All articles
SecuritySeptember 28, 20267 min read

The hallucination trap: page-accurate citations matter more than model size

A very large model that invents a clause is more dangerous than a small one that refuses to answer. How grounding, citation, and refusal should be tested.

Written for: Risk leads, technical evaluators, professional service firms

In a demo, a fabricated answer looks identical to a correct one. It is fluent, specific, and confident. That is precisely why model size is the wrong thing to shop for when the output will inform a contract, a diagnosis, or a board paper.

Three properties worth more than parameters

  • Grounding: the answer is built only from passages retrieved from your documents, not from the model's general memory.
  • Citation: the answer names the document and page, so verification takes seconds rather than an afternoon.
  • Refusal: when the documents do not contain the answer, the system says so instead of guessing.
Refusal is a feature, not a limitation

A system that answers everything is a system that answers some things wrongly. In professional work, "the documents do not cover this" is the correct and useful answer.

How to test a document AI in twenty minutes

  1. Upload one document you know intimately.
  2. Ask five questions whose answers you can verify on the page. Check every citation, not just the answer.
  3. Ask two questions the document does not answer. A guess here disqualifies the tool.
  4. Ask a question whose answer appears twice with different numbers. See whether it notices the conflict.
  5. Delete the document. Then ask the same questions again. If the old content still appears, deletion is cosmetic.

That last test is the one most tools fail quietly, and it is the one that matters when a client asks you to confirm their file has been removed.

A note on accuracy

AI Box answers from the documents you upload and shows the source document and page. Answers should still be checked against the cited page before they are relied on for legal, medical, or financial decisions.

Try it on a document of your own

Ask a sample document a question and see the answer with the exact page it came from — then configure the box your team needs.

Keep reading