If you take one practice away from this section, take this one: never let the same AI session that wrote the code also write and judge the tests for it.
Why a single session cannot check its own work
It tests the intent, not the requirement
An agent that just built a feature is holding a model of what it meant to build. When the same session writes tests, it writes tests against that model. If the implementation misunderstood the requirement, the tests inherit the misunderstanding exactly — and pass.
You end up with a green suite that proves the code does what the code does. That is a tautology, not a test.
Self-preference bias is measurable
Research into using language models as evaluators has consistently found a self-preference bias: models rate their own output more highly than equivalent output from a human or another model. Reported effects range widely but are substantial across every major commercial and open-source model studied, and — importantly — the bias is most harmful precisely when the model got it wrong, because that is when an inflated self-rating matters.
Part of the mechanism appears to be mundane: models score text with lower perplexity more generously, and a model's own output is exactly the text it finds least surprising. Familiarity reads as quality.
A tester whose judgment is systematically kinder to one particular author is not a tester.
Shared context contaminates the check
Even setting bias aside, the mechanics work against you. In one session the agent sees its own reasoning, its dead ends, its assumptions, the moment it decided a field was optional. All of that conditions the tests. The verifier pattern in multi-agent systems exists for this reason: an independent reviewer is given the requirements and the artifact, but not the generator's chain of thought — so it evaluates the output rather than re-validating the reasoning that produced it.
One failure becomes two
If the coding session is confused about the domain, that confusion now corrupts your implementation and your safety net at the same time. With separation, a wrong implementation meets tests that were written from the requirement and the failure surfaces immediately. Without it, both sides agree, everything is green, and the defect reaches production wearing a passing badge.
What separation actually means
Four levels, in increasing order of strength. Do as many as you reasonably can.
| Separate session | The minimum, and non-negotiable. A fresh session with no memory of how the code was written. |
| Separate context | The tester gets the requirement, the acceptance criteria and the running application — not the diff, not the implementation notes, not the build transcript. |
| Separate model | A different model has different blind spots. Where the builder's training pulled it toward one interpretation, the tester's pulls elsewhere. Disagreement is the signal. |
| Separate tool | Strongest of all. The tests live in Functionize and run on our infrastructure in a real browser, entirely outside the environment that produced the code. There is no shared state left to contaminate anything. |
That last row is the part worth dwelling on. Tests that are generated by the coding agent, into the coding agent's repository, and run by the coding agent's harness share a great deal of machinery with the thing under test. Tests that are described in Functionize Studio, stored in Functionize, and executed against your deployed application in a real browser share almost nothing with it. That separation is structural rather than a matter of discipline — nobody has to remember to enforce it.
Choosing a model for the testing agent
If you are using Anthropic models, a straightforward and effective arrangement is to have the builder and the tester be different members of the family. All of these connect to Functionize over MCP — see Getting Started with the Functionize MCP Server.
| Claude Opus 5 | The strongest reasoner. Use it for the test strategy conversation — deciding what to prove, picking candidates, reviewing a suite for gaps. Also a good adversarial reviewer of tests another model wrote. |
| Claude Sonnet 5 | The practical default for the testing session itself: turning acceptance criteria into Studio tests, running them and triaging what comes back. Fast enough to keep up with a build loop. |
| Claude Haiku 4.5 | Cheap and quick. Good for high-volume, low-judgment work — sweeping a suite, classifying a batch of failures, checking whether a set of tests still describes the documented behavior. |
A pairing that works well in practice: Opus 5 builds the feature, Sonnet 5 writes and runs the tests. Or invert it when correctness matters more than velocity — Sonnet 5 builds, Opus 5 tests adversarially, which is the arrangement to prefer for anything touching money, permissions or personal data.
Check Anthropic's current model list for the exact identifiers before you wire anything up; the names above are the families, and the specific versions move.
The principle matters more than the specific pairing: the tester must not be the same instance as the builder, and ideally not the same model. Two sessions of the same model with genuinely separate context is already most of the benefit. Two different models is better.
What the tester should and should not be given
| Give it | The requirement or user story · the acceptance criteria · the URL of the running application · test credentials from an execution preset · a skill describing your application's domain · known constraints ("orders under $10 are rejected") |
| Withhold it | The implementation diff · the builder's reasoning or session transcript · "here is how I solved it" · the names of internal functions, components or database columns · any hint about which edge cases the builder already considered |
The second column is the discipline. Every item in it is a way of telling the tester what to expect, and a tester who knows what to expect finds what it expected.
The objection, answered
"This is slower." It is slower per feature and considerably faster per quarter. The time you save by having one session do both is repaid, with interest, the first time a defect reaches production wearing a green badge.
"Our coding agent can already generate tests." It can, and those tests are useful as unit-level scaffolding written by the author. They are not independent verification, and treating them as such is the failure mode this page exists to prevent. Keep them. Add an independent layer above them.
"We do not have time to run two sessions." You do not run them serially. The testing session runs beside the build, against the deployed application, and reports on a schedule. That is precisely what an orchestration is for.
Where to go next
- Setting Up an Adversarial Test Loop With Claude and the Functionize MCP Server — the concrete setup, step by step.
- Reviewing AI-Generated Tests: What a Human Still Has to Check — the last gate, and it is a person.
- Why Context Matters, and How to Give Studio More of It — what the testing session should know about your domain.