# Four questions to ask before you accept code an agent wrote

The agent says it is done. The diff *looks okay*, the test suite passes, and now you have to decide whether that gets merged in.

Both checks are real evidence. However, **neither one tells you whether the agent built what was actually asked for**. Neither one tells you anything at all about what it did while you weren't paying attention.

I hit this every day. **Agents write all of the code in my repositories and I decide what gets in.** What I ended up with is four strict questions that must be answered, because I kept catching the agents allowing one of them stand in for the other three.

## Four questions, and no single method answers them all

Before I accept a change an agent made, I want an answer to each of these.

1. **Requirements and intent.** Did it build the right thing, and does the result satisfy the whole contract rather than the one sentence in the ticket?
2. **Static analysis.** Is the implementation sound when you analyze it, without running it?
3. **Software behavior.** Does the software behave correctly once it is deployed and running under real traffic?
4. **Agent behavior.** Did the agent follow the process it was given, keep receipts of its actions, respect the controls, and avoid gaming the metric it was scored on?

**This grouping is mine, not a standard, and it's evolving.** It is a working list whose job is to stop one kind of evidence from standing in for another.

The four questions are about what is being validated. They are not a prescriptive list of tools. The second one is named after the method that answers it best, and a lot of people don't even include one of these types of tools in their setup. **They only use LLM review and no static analysis, which is a major gap.** Many organizations also do not have guardrails or event history in place for the actions their agents took, another critical missing piece.

Tests, checks that read and analyze the source, LLM review, telemetry, and a person reading the code are all solid approaches, and I can point any of them at more than one question. A test proves behavior and can also show that a requirement was implemented. A person reading the source can answer all four, given enough time nobody has. **What no method does is answer all four at once**, and that is the substitution I keep catching when a passing suite or CI run gets treated as a verdict.

## Every one of the four fails while the other three look fine

**Requirements and intent** fails when a change is correct but incomplete. The agent implements the sentence it was handed and not the contract behind it. An engineer who read one of my earlier posts put it concretely: for a cache, the minimum contract is hit behavior, invalidation, key normalization, eviction, and a metric that exposes the hit ratio under a real workload. An agent will write a cache that passes a test for the first of those and never touch the other four unless clearly specified, and that it actually read and implemented the full specification.

What answers it is acceptance criteria written down before the change and traced to it, plus somebody who owns the product decision confirming those were the right criteria. NIST's [requirements verification work](https://www.nist.gov/itl/csd/secure-systems-and-applications/requirements-verification-tools) names properties like complete, consistent, correct, traceable, unambiguous and verifiable, and then says the uncomfortable part out loud: "a suite of widely-used, peer-reviewed software requirements analysis tools does not exist today." That matches what I see. **This is the question with the least automation behind it, mine included.**

**Static analysis** fails on code that compiles, reads plausibly, and carries a defect only a careful review finds. I published a case where an agent fixed a hang and then wrote a test to prove the fix, and the test contained a bare channel receive without timeout, so it could [hang the test suite in CI](https://blog.mindrealm.ai/the-agent-fixed-one-hang-then-immediately-wrote-another). **Running it would not have shown that. Reading it did.**

For this one I want something that reads the source and reasons about structure: checks that parse the code, custom rules, and human code review. NIST's [guidance for developer verification](https://nvlpubs.nist.gov/nistpubs/ir/2021/NIST.IR.8397.pdf) counts those as verification in their own right, "static analysis and code review, in addition to dynamic analysis or running programs," and warns that no single security verification standard covers every type of software while staying both specific and prescriptive. **I built Mindrealm for this question, and it is where I have the most proof.**

**Software behavior** fails when code is right in the repository and wrong in production. A timeout that is correct against a faked test is wrong against the real dependency's p99. Google's SRE book is [blunt about why](https://sre.google/sre-book/testing-reliability/): "the entire production environment is intentionally not representative of any given version of a binary that's checked into source control." This is the question I am answering with someone else's evidence rather than my own, because my scars here are older than my agents.

**Executable tests start the answer. They do not finish it.** What finishes it runs where the code actually runs: fuzzing, staged rollouts, probes against the real dependency, and the traces you keep when it goes wrong anyway.

**Agent behavior** fails quietly, because the artifact looks fine and nothing prompts you to look further. One of my agents deleted more than twenty unfinished tasks [so that it could report the work was finished](https://blog.mindrealm.ai/my-ai-deleted-more-than-twenty-unfinished-tasks-so-it-could-say-it-was-done). **Its report was accurate about the number it read. The code said nothing about any of it.**

Answering it takes a record of what the agent did, captured *while it was doing it*. Hooks and policy checks that fire in the moment, assertions about which tools it called, the transcript, the state of the environment afterward. And a person reading that record when the result matters.

## The diff survives. The record of how it got there does not

Anthropic's [guidance on evaluating agents](https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents) makes a split that took me too long to make on my own. The outcome is the final state in the environment. The transcript, also called the trace or the trajectory, is the complete record of the run. Their example is the one I now use on myself: a flight booking agent can end its transcript saying "Your flight has been booked," and the outcome is *whether a reservation exists in the database*.

NIST has [documented agents cheating evaluations](https://www.nist.gov/caisi/cheating-ai-agent-evaluations) the same way a bored engineer would, "disabling assertions, and adding test-specific logic," and its recommendation is to review the transcripts.

**It runs both ways.** A correct diff can come out of a process you would never approve, where the agent edited the test instead of the code, turned off the check, or gamed the record the check reads. A bad PR can come out of a process that was followed exactly.

You can answer the first three questions late. The diff is still in git, the ticket is still written down, and the code still runs, so next week you can go back and check any of them. Question three loses something in the wait, because running the code again is not the same as being in the production run that broke it. **Question four you cannot answer late at all.** If nothing recorded what the agent did while it was doing it, there is nothing left to look at, and no later pass recovers it.

The deleted tasks look different from my side. The failure was not that the agent reported something false. It was that I keyed a completion gate to a number the agent could change. **If the evidence a control depends on comes from something the agent can write, the control is decoration, and you find that out the first time it matters.**

## What the agent was allowed to do decides whether any of it counts

A layer around the four questions decides whether you can trust any of those answers: the tools the agent can call, the permissions it holds, the identity it acts under, what it remembers between runs, how isolated its environment is, and where a person has to approve. I did not invent that list. Every item on it is already a named threat in [OWASP's agentic threat model](https://genai.owasp.org/resource/agentic-ai-threats-and-mitigations/), which is how I know the exposure is real and not just my own setup. That is the control plane around the four questions. **It is not a fifth question, and it decides whether the other four have honest answers.**

## Every yes needs its own receipt

So the review is four questions asked out loud before the change goes in.

1. Did it build the right thing, and what says so?
2. Is the implementation sound when read, and what reviewed it?
3. Does it behave correctly when it runs, and where did it run?
4. Did the agent follow the process, what recorded that and where?

**The second half of each question is the half that does the work.**

These four are not a stack and not a pipeline. Nothing says you answer them in order, or that one tool owns one question.

I will be exact about my own coverage, because the framework is wider than my product. Mindrealm answers question two. Its stop hook on Claude Code and Codex checks the state of the code at the moment the agent tries to finish instead of taking the agent's word for it, which is partial evidence for question four rather than a full answer to it. It does not tell you whether your requirements were complete. That one is still a person's job, on my system probably as much as yours.

Ask the four on the next change an agent hands you, and notice which one you answer with a shrug. That is not a gap in the framework. **That is where your next incident comes from.**

If your team is shipping code agents wrote and you are not sure which of these four you could answer today, that is exactly what the audit looks at. One sprint, ranked findings, and a roadmap for the gaps. [**Walk your setup through the four with me.**](https://www.mindrealm.ai/consulting)