Knowledge that isn't there

Agents hallucinate when the knowledge they need isn't there. A model predicts a plausible continuation, and an empty space where the answer should be doesn't read to it as "I don't know" — it reads as room to generate. So instead of stopping at the gap, it fills it. Ask it something well documented and it does fine. Ask it something that was never written down, or was written three times in versions that disagree, and it produces a confident answer anyway. That confident-but-wrong output, acted on without a wide filter to catch it, is the middle of the barbell.

In most organisations the knowledge isn't there a lot of the time. The wiki is stale, two policies disagree, and the rule that actually governs a case lives in one person's head. People cope with it without noticing, or by wilfully ignoring contradictions to get things done — they know which page to ignore and which Slack thread holds the real answer. The agent has none of that. It reads what exists, and where nothing reliable exists, it fills in.

What has changed is that LLMs make writing knowledge down and looking it up much faster and cheaper. A process that lived in someone's head can be captured by interviewing them and having the model draft it. A pile of documents can be read across in seconds, with the gaps and contradictions pointed out. Both halves — getting knowledge written down, and finding it again later — used to be slow and expensive, and both have dropped in cost. (Producing the material is what got cheap; confirming it's correct still takes work.)

The natural consequence is to build systems optimised for learning. Not a one-off project to clean up the knowledge and declare it fixed — that always failed, because knowledge drifts the moment the project ends. Something ongoing instead: capturing, finding, and correcting what the organisation knows as normal background work, cheap enough to run continuously rather than as an occasional campaign. The organisations that get value from agents will be the ones whose knowledge is written down and current, which is now affordable to keep up in a way it never was.

The agent isn't really the point. It shows where the knowledge is missing, and it arrives just as the cost of fixing that has come down.

If you're thinking about agents and the knowledge underneath them looks thin, that's the interesting conversation. Reach out — happy to think it through.