veritas-gate — I Built a Hallucination Detector. It Scored 5 out of 100. I Published It Anyway.

I built a tool that checks AI-generated text for made-up claims. Then I tested it against a public benchmark. It scored 5 out of 100 — and a “detector” that does nothing but answer “yes, that’s made up” to every input scores 52.

I put that number at the top of the README instead of burying it. Here’s the whole story, because the bad number taught me more than a good one would have.

What the tool is

veritas-gate is a small Python library that checks AI-generated text against facts you know to be true. You give it two lists: an evidence bank (things the text is allowed to claim, because they actually happened) and a set of forbidden claims (things it must never say). Any sentence that contradicts them gets flagged, and the text is blocked instead of sent.

I built it to gate AI-drafted résumés and cover letters. In that setting one fabricated claim — a skill never used, a project never built — costs more than a hundred wrongly-blocked drafts. Inside that job, with its lists configured, the tool earns its keep.

The open question was the one every tool like this skates past: how well do these rule-based checks work outside the domain they were configured for? Most projects in this space just assert an answer. I measured one.

The test

There’s a public benchmark for this called RAGTruth: 2,700 AI-written answers where human annotators went through line by line and marked every fabricated statement. That gives you an answer key. Run your detector over all 2,700, and you learn two things:

• When it raises its hand, is it right? Mine was right 50% of the time — a coin flip, though the answers are only about 35% fabricated, so a coin flip is actually above chance here.
• Of all the fabrications in the dataset, how many did it catch? Mine caught fewer than 3 in 100.

Those two numbers combine into a standard 0-to-1 score (called F1 — high only when a detector is both usually right and catches most of what’s there). Mine: 0.05. The lazy strategy of flagging everything — every answer, no reading required — scores 0.52, because it trivially catches 100% of fabrications and is right 35% of the time. For reference, the benchmark’s own paper reports 0.53 for asking GPT-3.5 to judge, 0.63 for GPT-4, and 0.79 for a model trained specifically on this task.

So: my detector, on general text, loses to not even trying. That is the honest headline, and no amount of tuning would fix it. The reason is structural, and it’s the interesting part.

Why it fails — and why that can’t be tuned away

The two checks I could fairly run on general text are both, at bottom, number checks: they fire when the AI’s answer contains a percentage, a dollar figure, or a count that has no support in the source material. That works well on résumés, where the dangerous lies are inflated metrics.

But I counted what the fabrications in RAGTruth actually look like: only about 1 in 5 contains a number at all. The rest are invented names, made-up relationships, events that never happened. A number-checker is blind to all of it — not because the rules are badly written, but because of what the rules are. The ceiling is built into the approach.

I could have “fixed” the score by reading the benchmark’s answers and writing new rules to match what I found there. That’s cheating — building the detector out of the answer key — and it’s exactly the failure this measurement exists to catch. So the vocabulary shipped untouched, and the score is what it is.

The part nobody puts in the brochure

While building the comparison table I noticed something. The widely-cited GPT-3.5-as-judge baseline scores 0.53. Flag-everything scores 0.52. A famous published baseline beats “answer yes every time” by one point.

So the useful habit isn’t “distrust my tool” — it’s always ask what the do-nothing baseline scores before being impressed by any detection number. Including mine.

The second negative result

The same repo documents another design I killed with measurement. The idea: replace hand-written checking code with a simple table of banned phrases — easy to read, easy to extend. Before committing to it, I tested it against 9,963 real generated documents.

A two-character entry in the table (rl, a forbidden acronym) matched 3,410 documents. The properly-written version of the same check matched 107. The table version was firing on the letters r-l inside world, girl, early, hourly, and quarterly — a 31.9× false-alarm blowup, over three thousand false positives from one entry. And converting a numeric check into table form missed 112 real violations of the exact kind that check existed to catch. Wrong in both directions at once.

The table stayed — but only for multi-word phrases that can’t hide inside other words. Everything needing word boundaries or actual logic stays in code. That boundary isn’t a limitation I plan to fix later. It’s the finding.

What the tool is actually good for

One number did hold up: speed. A check runs in about a third of a millisecond, with the same result every run — the output file is byte-identical across repeated runs, and anyone can reproduce it with two commands. Asking an AI model to judge the same text takes 1–3 seconds and can change its mind.

That’s the honest scope: a cheap, instant first line of defense inside a domain you’ve configured — it catches the specific lies you told it to watch for, at effectively zero cost, before anything slower and smarter runs. It is not a general hallucination detector. Now there’s a public number that says so.

Why publish a failing score

The alternative is worse. This field is full of READMEs that say “catches hallucinations” above an evaluation that was never run — or was run once, disappointed, and quietly filed away. Anyone adopting my gate expecting a general detector would have discovered the 5/100 themselves, in production, with less instrumentation than I had.

And the measurement rig is the real deliverable: a fetch script so anyone can pull the benchmark and reproduce the numbers, proper statistical error bars, identical output on every run, and a README that says plainly what was not measured. Building the harness that produces an honest number is the transferable skill. The number itself just happened to be small.