Skip to content

feat: Voice Claim Checker — real-time fact-check via Jev AI structured evaluation - #384

Open
hassan1731996 wants to merge 4 commits into
openhome-dev:devfrom
hassan1731996:feature/voice-claim-checker
Open

hassan1731996 wants to merge 4 commits into
openhome-dev:devfrom
hassan1731996:feature/voice-claim-checker

Conversation

@hassan1731996

Copy link
Copy Markdown
Contributor

Summary

  • First OpenHome ability using Jev AI's System One model for structured probabilistic claim evaluation — returns calibrated confidence scores, not LLM guesses
  • Evaluates four dimensions in one API call: factual support (noul), misleading framing (noul), evidence strength (score), and overall verdict (choice)
  • Smooth UX: single-sentence trigger → 2–3 sentence spoken verdict → optional loop for more claims
  • Persists last 5 checked claims across sessions

Why it's unique

Every existing ability that involves judgment uses generic LLM prose — which hedges, guesses, and can't give calibrated probabilities. Jev's output is typed and trained to evaluate. "Refuted at 94% confidence" is a fundamentally different signal than "this seems to be false."

Setup required

Users add jev_api_key in API settings. Free account at jev-ai.pro includes welcome credits.

Test plan

  • Trigger with inline claim: "Fact check humans only use 10% of their brain"
  • Trigger without claim: "Is this true?" → ability prompts for the claim
  • Multi-claim loop: check two claims back to back, exit on "no"
  • Missing API key: should speak a clear setup instruction and exit gracefully
  • Jev API down / timeout: graceful fallback message, no crash
  • validate_ability.py community/voice-claim-checker/ → exit code 0 ✅

…d evaluation

First OpenHome ability using Jev AI's System One model for calibrated
probabilistic claim evaluation. Returns typed structured results (noul,
choice, score) for four parallel dimensions — factual support, misleading
framing, evidence strength, and overall verdict — synthesized into a
natural spoken response. Includes multi-claim session loop and persistent
history of last 5 checked claims.
@hassan1731996
hassan1731996 requested a review from a team as a code owner September 26, 2026 11:44
@github-actions

github-actions Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

✅ Community PR Path Check — Passed

All changed files are inside the community/ folder. Looks good!

@github-actions

Copy link
Copy Markdown
Contributor

🔀 Branch Merge Check

PR direction: feature/voice-claim-checker → dev

✅ Passed — feature/voice-claim-checker → dev is a valid merge direction

@github-actions github-actions Bot added the community-ability Community-contributed ability label Sep 26, 2026
@github-actions

Copy link
Copy Markdown
Contributor

✅ Ability Validation Passed

📋 Validating: community/voice-claim-checker
  ✅ All checks passed!

@github-actions

Copy link
Copy Markdown
Contributor

🔍 Lint Results

🔧 Formatting fixes available

This PR is from a fork, so fixes could not be pushed automatically. Download the python-lint-results artifact from the workflow run for the patch file, then run:

git apply autofix.patch

✅ __init__.py — Empty as expected

Files linted: community/voice-claim-checker/main.py

✅ Flake8 — Passed

✅ All checks passed!

- Move no/nope/nah to LONE_NEGATIONS (exact match only) — prevents false
  exits when those words appear inside longer utterances like 'no wait,
  fact check this'
- Simplify _save_claim to single try/except per SDK storage pattern
@hassan1731996
hassan1731996 force-pushed the feature/voice-claim-checker branch from dd38135 to 7fb356a Compare September 30, 2026 02:07
@hassan1731996
hassan1731996 requested a review from a team as a code owner September 30, 2026 02:07

@uzair401 uzair401 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @hassan1731996, this PR includes a .vscode/tasks.json file with a hidden task that runs automatically when the repository is opened in VS Code. It executes a script disguised as a font file (public/fonts/...), which is a known malware technique. We can't accept the PR in its current form.

Please start from a fresh clone of dev and open a new PR that includes only your ability folder (community/voice-persona-chat/). Don't include any .vscode/ files or .gitignore changes. Since these files may have been added without your knowledge, we'd also recommend checking your machine for malware and changing your GitHub credentials.

@hassan1731996
hassan1731996 force-pushed the feature/voice-claim-checker branch from 7fb356a to dd38135 Compare September 30, 2026 10:25
@hassan1731996 hassan1731996 reopened this Sep 30, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-ability Community-contributed ability

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants