A rule in a prompt is not a guarantee
The first version put the whole anti-hallucination protocol, a chain-of-verification routine and the three knowledge tiers into the system prompt of every call. It was long, costly, and nothing confirmed the model had followed it.
Why it was hard, what I decided, how AI helped
- Why it was hard
- Asking a model to verify its own draft in the same answer cannot be observed from outside. Making the prompt longer only adds cost on every call.
- My approach
- I kept the trust rule fixed: no invented statistics, legal claims, dollar amounts or agent stories. I asked for the checking to move out of the prompt and into steps that leave a trace.
- How AI helped
- Claude Code condensed the protocol into a short data-integrity section of the core prompt and moved the full rulebook into a reference document. It added code checks after the draft and a separate fact-check step with its own model calls.
- The decision
- Keep the principles in the prompt and put the checking in code and in separate calls, so each check either runs or visibly does not.
- Evidence
- commit 623e8a5 · lib/prompts-core.ts · docs/prompt-reference.md · app/api/generate/route.ts


