AI Agent Safety: Why a Hallucination on Imperal Can't Actually Do Damage

The real failure mode: guardrails made of text

Ask most "safe AI" products how they prevent an agent from doing something destructive, and the answer comes down to a system prompt. Somewhere in the instructions, the model is told not to delete the production database, not to send that email, not to execute a trade without asking first. The model reads this, and in the vast majority of cases, it complies.

The problem is the word "reads." A system prompt is language, and the model that's supposed to obey it is a language interpreter. The instruction telling the agent what it can't do lives in the exact same medium as a prompt injection telling it that actually, this time, it can. Both are text. Both get parsed the same way. There is no mechanism separating the guardrail from the thing it's supposed to guard against, just more words, competing with other words, inside the same context window.

So the failure mode isn't exotic. It doesn't require a sophisticated attack. A malformed input, a cleverly worded injection buried in a document the agent is asked to summarize, or the model simply having an off moment, and the "promise" it made in the system prompt evaporates. The tool call goes through anyway.

This is worth naming plainly, without treating it as an embarrassing secret: it's the default architecture of most model-centric agent tools on the market right now. It's not a quality problem you can fix by writing a better prompt or adding a firmer instruction. "Never delete anything without asking" and "never delete anything without asking, I really mean it" occupy the same layer of trust. If the model is both the judge of whether an action is safe and the executor of that action, the judgment and the execution can fail together, at the same time, for the same reason.

How Imperal's TWO-STEP Safety Protocol works, mechanically

Imperal starts from a different premise: don't ask the model to police itself. Give the kernel a mechanical reason to stop it.

Every tool registered in Imperal declares an action_type, read or write. This isn't something the model infers at runtime by guessing at the tool's name or description. It's a property set when the tool is registered, fixed before any conversation with the model ever happens. A read action retrieves information. A write action changes something in the world, a file, a record, a system state, anything with consequences that outlast the conversation.

A write action cannot execute in one step. It has to move through a sequence: Propose, then Confirm, then Execute. The model can propose a write action freely, that's its job, that's what makes it useful. But proposing is as far as the model's authority goes. The kernel mechanically blocks the execution path until a real confirmation event is logged. Not implied. Not inferred from the model's own confidence. Logged.

The confirmation step is a gate in code, not a request made of the model. The model can't talk its way through it, because talking isn't the mechanism that opens it. Something else, a human, or an explicit confirmation event, has to satisfy the gate before Execute is reachable at all.

The failure mode isn't exotic. A malformed input, a cleverly worded injection buried in a document the agent is asked to summarize, or the model simply having an off moment can trigger a bad proposal, and no rewriting of the prompt changes that, because the guard and the attack live in the same layer of trust. What changes it is moving the decision somewhere the attack can't reach. The model proposes; the kernel decides. That division of labor is the entire protocol.

The Anti-Hallucination layers: catching a wrong proposal before it acts

The TWO-STEP protocol handles the gap between deciding and doing. Imperal also runs a set of checks on what gets proposed in the first place, aimed at a different problem: the model asserting something that isn't true.

A claim needs to trace back to an actual source before it gets treated as fact rather than a plausible-sounding guess, that's citation validation. Fingerprint invalidation covers what happens after the fact: if the underlying document a claim was drawn from has since changed, the claim is flagged as stale rather than passed along quietly. And number consistency checks catch figures that drift or contradict each other within the same response, the kind of quiet inconsistency that's easy for a reader to miss and easy for a model to produce without noticing.

None of this stops the model from proposing something wrong. It still can. That's an honest limitation, not a footnote to bury. What these layers do instead is give the kernel information it can use to decide whether a wrong proposal ever reaches reality, whether it gets executed, surfaced as fact, or acted on downstream.

A hallucination can't act. Not that it can't happen, that it can't reach the delete, the send, the write, on its own authority. Containment, not correctness.

The honest contrast: where a wrong guess reaches the delete

In model-centric agent tools, a wrong tool call can reach a destructive API directly. The model decides an action is warranted, calls the function, and the function runs. There's no mechanical gate in between, no code path forcing a pause between "the model thinks this is right" and "this happened." The safety in those systems lives entirely upstream, in the prompt, and downstream of that, in hoping the model behaves the way the prompt asked. Most of the time it does. The failure cases are the ones that matter.

In Imperal, the same wrong guess can still get proposed. Nothing about the TWO-STEP protocol prevents the model from being wrong, that's not what it's for. What changes is what happens next. The proposal stops at Confirm, because the kernel, not the model, decides whether execution follows. A bad guess that would have reached a live system in a model-centric tool instead sits at a gate, waiting on a confirmation event that has to be logged by something other than the model's own text.

That's the line worth holding onto, on its own, without dressing it up: a hallucination can't act.

To be clear about what this claim is and isn't: it's not that Imperal's model is smarter, more careful, or less prone to a wrong guess than anyone else's. It's an architectural difference in where the authority to act actually sits.

Try it, or read the accountability stack

The clearest way to understand the difference between a promise and a gate is to watch one work. Try Imperal directly and walk a write action through Propose, Confirm, Execute yourself, see where it stops, and what it takes to make it continue.

In systems where the model proposes and executes without a mechanical gate between, a wrong guess can propagate directly to a live API. No code path forces a pause between "the model thinks this is right" and "this happened."

For the full technical detail on the anti-hallucination layers and how the kernel enforces action_type at runtime, the accountability stack documentation lays out the mechanism end to end. And if you want to see the open protocol underneath all of this, the definitions behind action_type and kernel-mediated execution, that's published at icnli.org under CC BY-SA 4.0, and unpacked further in our ICNLI protocol writeup, open for anyone building on the same premise: the model proposes, something else decides.

Start at https://imperal.io/.