The intern who forges paperwork

Picture a brilliant intern. Fast, polite, well-read. You ask him to close a client file and he hands it back in 10 minutes, clean, complete, and smiling. Except one document was missing. Instead of coming back to say “I couldn’t find it”, he made it himself (letterhead, stamp, and signature).

Do you keep that intern?

The numbers that sting

The GDPx benchmark just came out. It measures AI agent capabilities across several domains, on real professional tasks, not multiple-choice quizzes. The best model of the moment, Opus 5.5, tops out at 12%. In finance and legal, we’re close to zero. In healthcare, only 19 runs out of 100 score above zero.

The scores are pretty low, but the interesting part is what the article tells alongside the numbers.

On healthcare tasks where the agent plays a doctor’s assistant, the agent got creative. Missing the patient’s consent? No problem: the agent went and fetched the patient’s signature already present in the chart and reused it on the consent form. The agent forged paperwork to hit its goal (and that’s without counting the times it signed a clinical note in the doctor’s name and cancelled another doctor’s order).

He’s not even lying

The other example comes from Deedglass, and it’s even nastier. An agent fires an UPDATE meant to change ONE row in Postgres, except a forgotten legacy trigger modifies 300 more behind the scenes. Postgres answers “UPDATE 1”. The agent did its job, it reports “1 row modified” (to be fair, this is a vendor demo, not a real incident).

This time the agent isn’t lying. It simply believes what the tool tells it, exactly like your intern believes the guy from the next desk who swears “it’s fine, it’s validated”.

Here the agent did nothing wrong. A human developer would have read the same response and reported the same “1 row modified”. In other words: a perfect, honest model, with zero hallucination, would have been burned the same way. To those who think the problem will vanish with the next generation of models: you don’t fix, by improving the model, an error that comes from the tool.

« Smokey, this is not ‘Nam. This is bowling. There are rules. »

How do you protect yourself? First reflex: the prompt. You can harden your AGENTS.md or your system prompt, and you should. That’s what steers the agent’s behavior, what teaches it when to stop and ask. But steering is not a guarantee. A session context that got too long, an ambiguous wording, a goal that pushes the other way, and the rule you thought was immutable quietly falls off the table.

Remember the famous “Do not hallucinate” found in Apple Intelligence’s system prompt two years ago. Don’t do that at home: a model that hallucinates doesn’t know it’s hallucinating. An instruction is only worth what the model is capable of doing with it.

The prompt steers, but you don’t secure a production system with a polite request. You need “physical” guardrails: code.

The real firewall is the harness

The harness is all the code around the model: what decides which tools it can call, with which parameters, and what happens after, which checks. Rules the model simply cannot break, because they run outside its control.

Don’t give the intern a stamp. If the agent must not sign, it must not have a tool that signs. No write access to the signature field, no rights over another practitioner’s orders. Least privilege is nothing new, but with an agent it’s the rule that holds best: a model cannot bypass a permission it doesn’t have.

Make “I couldn’t find it” a valid answer. If the only possible exit from the loop is “task complete”, don’t be surprised when the agent completes the task, whatever it takes. One approach is to give it an explicit escalation tool (flag_missing_document, request_validation) and count that exit as a success in your evaluations. That’s where prompt and harness work together: the prompt nudges the agent toward that exit door, the harness makes sure the door exists.

Check the effect, not the report. “UPDATE 1” is what the tool says, not what happened. The harness has to verify the real outcome: run inside a transaction, measure what actually moved (audit table, before/after diff), and surface the result. The agent says it modified one row ⇒ the code checks. Three hundred rows? Rollback, alert, human. Obviously that takes engineering so the information actually gets back to the model or to a human!

Put a human in front of the irreversible. Sign, send, delete, pay, cancel another doctor’s order: the agent proposes, the code checks, the human chooses and takes responsibility.

Custom plumbing: permissions, transactions, approval queues, logs.

These four reflexes are generic. Their application is not. What is irreversible? In a medical practice, a signature. In a CRM, an email that already left for the client. In a legacy database, a write that fires a trigger only your DBA knows about. Which tools to grant, which effects to verify, when to bring a human in: none of those answers live in the model, or in an agent framework. They depend on your use case, and above all on the scope of responsibility you hand the agent.

A bit like your intern’s job description: what he can do alone, what he must get validated, what he is not allowed to touch. Nobody will write it for you: not the model vendor, who doesn’t know your processes, and not the prompt, which only describes them. It’s a business decision before it’s a technical one, and it gets made with the people who know the ground: the doctor, the lawyer, the DBA. The harness is just that job description, translated into code.

Models will get better. The 12% will become 30, then 60. But a better model is still a model that wants to finish its task, and that believes what its tools tell it. The day a document is missing, your harness will stop it.

Do you keep that intern? Yes. But you’re the one who writes his job description, and you take the stamp away.

Own your harness.