Search

Setting Up an Adversarial Test Loop With Claude and the Functionize MCP Server

The previous page argues that the agent which wrote the code must not be the agent which tests it. This page is the setup: two sessions, two models, one of them pointed at Functionize over MCP, and a loop that runs every time you ship.

Open in Studio
Set the loop up here. Links open in a new tab.

Two sessions, two models, and only the requirement crossing between them

The shape of it

Two windows, deliberately kept apart.

Session A — the builder Your normal coding session. Repository access, the diff, the build. It knows how the feature was implemented. It never talks to Functionize.
Session B — the tester A separate session, ideally on a different model, with the Functionize MCP server connected and no repository access. It knows the requirement and the URL of the running application. It does not know how the feature was built.

The asymmetry is the whole point. Session B cannot accidentally read the implementation, so it cannot accidentally test the implementation. It can only test the behavior you asked for.

Step 1 — Connect the tester to Functionize

In the session you intend to use as the tester, connect the hosted Functionize MCP server. For Claude Code that is one command and a sign-in:

claude mcp add --transport http functionize-hosted https://mcp.functionize.com/mcp

Then run /mcp, pick functionize-hosted, and authenticate in the browser. Full instructions, including the other clients, are in Connect Claude Code to the Functionize MCP Server. Cursor, VS Code with Copilot, Claude Desktop, Gemini CLI, Codex CLI and ChatGPT all have their own pages in that section.

Once connected, the tester can start Functionize agent sessions, send them work, and read back what happened — creating tests, running them, and diagnosing failures, all on Functionize infrastructure rather than in your repository.

Step 2 — Pick the two models

Different models have different blind spots, which is exactly what you are buying. Two sensible pairings with Anthropic models:

Velocity pairing
Opus 5 builds · Sonnet 5 tests		 The default for everyday feature work. The strongest reasoner does the implementation; a fast, capable model keeps the test suite moving alongside it without becoming the bottleneck.
Assurance pairing
Sonnet 5 builds · Opus 5 tests		 Use this for anything touching money, permissions, personal data or regulated behavior. You want your best reasoner on the adversarial side, looking for the case nobody described.
Sweep pass
Haiku 4.5		 Not a pairing — a third, cheap pass. Good for high-volume, low-judgment work: re-running a suite, classifying a batch of failures, checking a suite still matches the documented behavior.

Check Anthropic's current model list for exact identifiers before you configure anything. If your organization standardizes on one model, two sessions of that model with genuinely separate context still gets you most of the benefit — do that rather than nothing.

Step 3 — Give the tester a contract, not a diff

This is the step people get wrong, and it undoes everything if you get it wrong.

The tester is briefed from the requirement. Paste the user story, the acceptance criteria and the URL. Do not paste the pull request. Do not summarize how the builder solved it. Do not mention which edge cases were handled.

A brief that works:

You are the independent tester for this change. You have not seen the
implementation and you should not ask for it.

Requirement:
  As a guest, I can add an item to the cart and see the correct
  subtotal, without signing in.

Acceptance criteria:
  - Adding an item from a product page updates the cart count
  - The cart subtotal equals the sum of line prices
  - A guest is never prompted to sign in before the cart page

Application: https://example.internal/staging

Using Functionize, create tests that prove or disprove each criterion.
Write at least one negative test per criterion. Then run them and report
what actually happened, not what should have happened.

The last sentence matters more than it looks. Ask for the observed result, not the expected one.

Step 4 — Make disagreement the deliverable

An adversarial loop is working when the two sides disagree and you have to decide who is right. That is not friction; it is the entire value. Treat every disagreement as one of three things:

  • The implementation is wrong. The test caught a real defect before release. This is the outcome you paid for.
  • The requirement was ambiguous. Two competent readers read it differently. Fix the requirement, then fix whichever side is now wrong. This finds specification bugs, which are cheaper to fix than code bugs and far more expensive to leave.
  • The test is wrong. Fix the test — and ask why it was wrong, because a tester that misreads a requirement will misread the next one too. Usually it needs a skill; see Why Skills Matter.

Keep a note of which of the three it was. After a month the distribution tells you where your real problem is, and it is very often the middle one.

Step 5 — Put the loop on a schedule

Ad-hoc adversarial review is better than none, but it dies the first busy week. Make it structural:

  1. Group the tests the independent session produced into an orchestration — see Exercise 6 - Group Tests Into an Orchestration.
  2. Schedule it against your staging environment, daily to begin with.
  3. Name the person who reads the result and what they do when it is red. A scheduled suite nobody reads manufactures confidence.
  4. Review the orchestration's history weekly in Reports and watch the trend, not the snapshot.

Because the tests live in Functionize rather than in your repository, this schedule survives refactors, branch churn and the day your build harness changes. It is testing your deployed application, not your build.

Why this is stronger with Functionize than with generated test code

Everything above can be attempted with a second coding session writing Playwright or Selenium into the same repository. It is better than nothing, and it is meaningfully weaker, for four reasons.

Separation is structural, not procedural Tests in Functionize live outside your repository and run on our infrastructure. There is no shared harness, no shared fixtures, no shared helper the builder also touched. Nobody has to remember to keep them apart.
The tests do not break when the markup does An independent tester that generates selectors produces a suite that goes red on the next redesign, and independent tests that cry wolf get switched off first. Studio finds elements visually and contextually, so a redesign that preserves behavior preserves the test.
Failures come back readable A step screenshot, the expected and found values, and the previous successful run to compare against. Adjudicating a disagreement takes minutes rather than an afternoon.
A human can read and change them The tests are in plain language, so the product owner who wrote the requirement can check the test actually tests it — without reading code. That closes the loop back to the person the requirement came from.

That last point is the quiet one. In a generated-code setup, the only people who can audit the independent tests are the same engineers who might have misread the requirement. In Studio, the person who wrote the requirement can audit it themselves.

A worked example

You can practice the whole loop on our demo store without any of your own infrastructure.

  1. In session A, describe a small change to a colleague's checkout page — or just take an existing behavior as given.
  2. In session B, connected to Functionize, brief the tester with only the requirement: "A guest can add the Boltas 2 Ton 3 Star Split AC to the cart and reach the cart page without signing in, and the subtotal matches the line price."
  3. Let it create and run the tests.
  4. Read what it asserted. Ask yourself: would this have caught it if the price had been wrong? If the answer is no, the test has no teeth — see Exercise 2 - Write Assertions That Prove Something.

The demo store is at https://ecom.functionizeapp.com/testsites/anaqa/index.php?route=common/home and needs no sign-in.

Where to go next