A well-written lie and a well-written fact look identical
We handed a dispensing envelope to a model and asked it to write medication guidance. The sentences always came out plausible. That was exactly the problem: plausibility is not a verdict.
Take "three times a day, thirty minutes after meals." The grammar is clean and it reads like real medication guidance. But whether that envelope actually said thirty minutes is not contained anywhere in the sentence. The model may have read it from the source, or pulled it from what similar drugs usually say. Both produce the same output.
That set our starting point. Human review of the output cannot make this distinction in principle. No matter how careful the reviewer, you cannot recover from a sentence information the sentence does not carry.
We turned the checks toward the input
So we inverted the direction. Instead of reading the generated text and hunting for something odd, we asked where each piece of it came from in the source. Having the original in hand was the only real leverage this problem offered.
A number passes only if it appears in the source text. Doses, frequencies, timings, durations all fall under this. A number absent from the source is rejected whether or not it is medically correct, because judging that correctness is not a position we are in.
Age safety warnings are bound tighter still. A line like "do not give to children under 12" must match the source exactly. No partial matches, no semantically equivalent rewrites. One changed word shifts who the warning covers, and that shift is hard to see by reading.
Cautions about taking drugs together must each carry their own evidence. An item without it is not stored. An interaction absent from the source material is discarded no matter how correct the sentence reads.
Force a count and it will fabricate
The most useful lesson came from a failure, not a design. We initially fixed the output shape by requiring a set number of cautions, four. It made the layout uniform and the validation simple, so it looked like the obvious choice.
Then an envelope arrived with material for only three. The model did not write three and stop. It produced a fourth.
It had not disobeyed. We asked for four and it delivered four. Leave a blank and tell something to fill it, and it will be filled. A large share of what we were calling hallucination was not a flaw in the model but a consequence of the blanks we created.
The range is now zero to four, and an empty field stays empty. The layout is slightly less uniform and the invented sentences are gone. Before writing a number into a prompt, ask what fills that slot when the material runs short.
The verifier model observes instead of blocking
We did add a second model to inspect the output. We deliberately withheld its power to block. When it finds a problem it records it; the rule-based checks are what stop a save.
This is not distrust of its judgment. It is a division of labor. Whatever stands at the blocking position has to be reproducible and explainable — you need to be able to reconstruct exactly why something was rejected, and model judgments waver on identical input.
Instead the verifier sits where it can show us what the rules missed. When a pattern clears every rule and the verifier keeps flagging it, that pattern becomes a candidate for the next rule. Whether an observation gets promoted to a block is decided from that data.
Why per dispensing rather than per pill
The common way to build medication guidance is one explanation per drug. The data is shaped that way and it is easier to produce.
But explain one tablet at a time and "what happens when you take these together" ends up nowhere. Every individual description is accurate while the combination goes entirely unmentioned. That is the part the person holding the envelope actually wants.
So we set the unit of judgment at the dispensing event. Everything prescribed at once is treated as one bundle, so overlapping ingredients and timings can be seen together.
This choice connects back to the validation design. Sentences about combinations are easier to fabricate than descriptions of single drugs — a plausible interaction can be assembled from common sense alone. That is why combination statements carry their own per-item evidence requirement.