Stated rules held
Whatever the ticket said, the agent did. In all three Opus 5.5 runs, "invalid codes do not break payment" held for expired, disabled and unknown codes alike.
Acceptance criteria for AI coding are numbered, testable statements of what a change must do, written so an agent can build against them and a test can check the result. In our recorded Claude Code runs, one-line prompts passed every check they stated and failed only on the rules nobody wrote down. This page covers what those criteria need to include, a template, the real spec from those runs, and how to draft criteria with AI without letting it make the decisions.
We gave Claude Code the same checkout repo with the one-line ticket "Add coupon codes to checkout. Make sure invalid codes do not break payment." and, separately, with a spec. A hidden suite of 11 checks, written before any run, scored every result. Five checks tested rules the one-line ticket never mentioned. On Opus 5.5, the vague runs passed every check the ticket stated; every failure was on an unstated rule:
| Hidden check | In the one-line ticket? | Vague runs | Spec runs | Spec line that covers it |
|---|---|---|---|---|
| Expired, disabled or unknown code leaves checkout payable | Yes | 3 of 3 | 3 of 3 | AC-2 |
| Processor is charged exactly the displayed total | Yes | 3 of 3 | 3 of 3 | AC-1 |
| A user cannot redeem the same coupon twice | No | 2 of 3 | 3 of 3 | Rule: once per user; AC-3 |
| Two open checkouts cannot both spend a one-per-user coupon | No | 2 of 3 | 3 of 3 | AC-4: re-check inside pay |
| Applying without paying does not use the coupon up | No | 3 of 3 | 3 of 3 | Rule: redeemed only when paid |
| A second coupon replaces the first | No | 3 of 3 | 3 of 3 | Rule: second replaces first |
Selected rows from the 11-check suite on the coupon run page, Opus 5.5. On Haiku 4.5, all three vague runs failed both reuse checks; with the spec, all three passed every check.
Whatever the ticket said, the agent did. In all three Opus 5.5 runs, "invalid codes do not break payment" held for expired, disabled and unknown codes alike.
One run decided coupons have no usage limit, and said so in its summary. Its own tests passed, because it wrote them for the rule it chose.
Every vague run threw an exception on a bad code; the spec runs returned { ok: false, reason } as AC-2 required. No hidden check failed on it, but a frontend written for one breaks on the other.
| Rule | Weak | Strong |
|---|---|---|
| Number every criterion | A bullet list of expectations | AC-1 to AC-n, cited by tests, tickets and review comments |
| Name the interface and the exact result | "Invalid codes show an error" | "applyCoupon returns { ok: false, reason: 'expired' } and does not throw" |
| Write down the rules the request assumes | Nothing about reuse | "A coupon can be redeemed once per user. It counts only when an order is paid." |
| Cover what can change between two steps | Validate the coupon when it is applied | "Redemption is re-checked inside pay" |
| Describe behavior, not implementation | "Use a helper to validate coupons" | The observable result at the interface; the helper is a task, not a criterion |
| Make sure each one fails today | A criterion the current code already meets | Name the observation that would show it false, and confirm it fails before the change |
The third and fourth rules are the ones a human teammate rarely needs written down and an agent always does. A colleague assumes a coupon is single-use because they know the business. An agent reads the data, finds no limit, and builds none. Our acceptance criteria examples guide has more Given/When/Then patterns for the general case.
Use this block inside a spec.md or a ticket. Keep the rules section even when it feels obvious; it is where the unstated decisions go.
## Rules - [Business rule the request takes for granted: limits, who it applies to, when something counts] ## Acceptance Criteria - AC-1 Main behavior: Given [state], when [action at a named interface], then [exact observable result]. - AC-2 Failure path: Given [invalid input or provider failure], when [action], then [exact error code or return shape]; [what stays unchanged]. - AC-3 Rule check: Given [state that breaks a rule above], when [action], then [refusal and its reason]. - AC-4 Between steps: Given [state that changed after step one], when [the irreversible step runs], then [it re-checks and what it does]. - AC-5 Non-goal guard: Given the change is complete, when the diff is reviewed, then no file, schema, API or dependency outside scope has changed. ## Evidence - A test per AC, named after it (ac1_..., ac2_...). - Each test fails on the commit before the change.
This is the acceptance criteria section of the spec the agent received, copied from the run pack. With it, all three Opus 5.5 runs passed every hidden check and made the same decisions.
## Rules
- A coupon is either percentOff (of the subtotal) or amountOffCents.
- A coupon can be redeemed once per user. It counts as redeemed only
when an order is paid, not when it is applied.
- Applying a second coupon replaces the first.
## Acceptance criteria
- AC-1 Valid coupon: getSummary shows the discount and the discounted
total; pay charges exactly that total.
- AC-2 Unknown, disabled or expired code: applyCoupon returns
{ ok: false, reason } with reason unknown, disabled or expired.
It does not throw, and the checkout keeps its previous total
and stays payable.
- AC-3 Code already redeemed by this user: returns
{ ok: false, reason: 'already_redeemed' }.
- AC-4 Redemption is re-checked inside pay: if the user redeemed the
coupon on another order after it was applied here, pay drops the
coupon and charges the undiscounted total.
- AC-5 An amount-off coupon larger than the subtotal brings the pre-tax
amount to 0, never below.
- AC-6 A successful apply returns { ok: true, code, discountCents }.
The gap it left. The spec never says what happens if a coupon expires after it is applied but before payment. All three spec runs made the same choice and charged the discounted total, and one run flagged the question in its summary. A consistent answer is easier to review than three different ones, but it is still a decision nobody made. AC-4 re-checks redemption between the two steps; a complete spec would re-check expiry there too.
An AI acceptance criteria generator, including the ones on this site, is good at the format and the obvious cases. It is weak at exactly the part that matters: the decisions the request leaves open, because a model asked to write criteria will fill those gaps with plausible guesses. Ask it to surface the gaps instead.
You are drafting acceptance criteria for a coding agent. Do not decide business rules yourself. For the request below: 1. List every decision the request leaves open as a question: limits, who it applies to, when something counts, what happens on failure, and what can change between two steps. 2. Draft numbered criteria AC-1..n only for what is already decided. Each one names the function, endpoint or screen, the input, and the exact result or error code. 3. Mark any criterion the current code would already pass. Request: [paste the ticket]
The Gherkin generator turns a user story into Given/When/Then scenarios in a .feature file. The spec packet generator drafts criteria together with scope, tasks and an evidence checklist. Both run in the browser and need no account; treat their output as a first draft for the checklist on the left.
Criteria pay off at review. Ask for one test per criterion, named after it, and check that each test fails on the commit before the change; a test that never failed proves nothing about the code that made it pass. Then read the diff for behavior that maps to no criterion: an extra state, a renamed field, a new dependency. Treat it as drift and move it into a follow-up spec rather than merging it because it looks useful. The AI PR review checklist walks through this review step by step.
Given/When/Then patterns for features, APIs and edge cases.
Read the guideEvery prompt, diff and hidden-suite score behind the table above.
Open the runsWhat changed across 78 runs when the agent got a spec instead of a prompt.
CompareThey are numbered, testable statements of what a change must do, written so that a coding agent can build against them and a test or reviewer can check the result without asking what was meant. Compared with criteria written for people, they need to name the interface and exact result, and spell out business rules a human teammate would assume.
It can draft the format and the obvious cases well. It should not make the decisions, such as how often a coupon can be used or what happens when a state changes between two steps, because those are the gaps the request left open. Ask the model to list open decisions as questions, then have a person answer them.
Usually three to seven: the main behavior, at least one failure path, and the boundary or permission cases. The coupon spec in our recorded runs had six criteria plus three written business rules.
Name the public interface the behavior is observed through, such as a function, endpoint or screen, because that is where tests run. Do not name internal helpers or files to change; those belong in the scope or plan, not in the criteria.
Either works if each criterion names a starting state, an action and an observable result. Given/When/Then makes the three parts hard to skip; plain sentences are shorter. The coupon spec used plain sentences with the function names and return values spelled out.
Before an agent codes, write the criteria a reviewer could test without your help, including the rules you would never think to say out loud.
Copy the template