Method comparison, tested

Vibe Coding vs Spec Coding

Vibe coding means prompting an AI agent in plain language and judging the result by whether it runs and looks right. Spec coding means writing down the outcome, non-goals, acceptance criteria and evidence first, and having the agent build against that. We ran both on the same three tickets, 78 times across four Claude models, and scored every run with hidden tests: with a spec, 28 of 30 runs passed every check; without one, 12 of 48 did.

Last updated: November 11, 2026

What vibe coding and spec coding mean

Vibe coding

Andrej Karpathy named it in February 2025: describing what you want to a model, running what it produces, and steering with follow-up prompts, to the point where you "forget that the code even exists". Collins named it its Word of the Year for 2025. The prompt is the only record of what was intended, and the model fills every gap the prompt leaves.

Spec coding

Also called spec-driven development or spec-first development. Before an agent writes code, a short spec states the outcome, what is out of scope, acceptance criteria someone can test, and the evidence that will prove the change works. The agent implements against the spec, and review checks the result against the same list.

They are not two kinds of tool. The same agent and the same model can do either. The difference is what the agent is told before it starts, and therefore who makes the product decisions: the model while it writes the code, or a person before the code exists.

Vibe coding vs spec coding at a glance

Vibe codingSpec coding
Starting pointA prompt in plain languageA short written spec: outcome, non-goals, acceptance criteria, evidence
Who decides product behaviorThe model, while it writes the codePeople, before the code exists
Time to a first versionFastestSlower by the time it takes to write the spec, usually minutes to an hour
Same request, run twiceCan come back as two different productsMatched each other in our coupon runs, even on a case the spec missed
What review checksDoes it run, does it look rightDoes each acceptance criterion pass
What is left afterwardsA chat log, if anyone kept itThe spec, next to the code or in the pull request
Typical failureUnapproved behavior ships because it workedOver-specification, and specs that drift from the code
Best forPrototypes, throwaway scripts, exploring optionsProduction behavior, money, permissions, user data, shared APIs

What 78 recorded agent runs showed

Most comparisons stop at opinions, so we measured. Claude Code got three ordinary tickets: add coupon codes to a checkout, clean up an API's error format, and split a name column in two. Depending on the ticket, it was run from a one-line prompt, from the same prompt with the repo's written rules removed, and from a spec, on four Claude models. A hidden test suite, frozen before the first run and never shown to the agent, scored every result. All prompts, diffs and scores are in the run packs.

ModelWith a specWithout a specAverage cost per run, prompt → spec
Opus 5.59 of 97 of 15$0.18 → $0.22
Sonnet 5.59 of 94 of 15$0.09 → $0.11
Haiku 4.57 of 90 of 15$0.10 → $0.20
Fable 5.1 (one run per variant)3 of 31 of 3$0.71 → $0.83

Runs that passed every hidden check, out of runs recorded. Costs are API-equivalent averages reported by Claude Code.

Vague prompts mostly work

On small, readable repos the one-line prompts fixed crashes, capped discounts and backfilled data correctly. If the only question is "does it run", a current model often passes it. That is why vibe coding feels fine until it does not.

The spec removes the variance

With a spec, the smallest and largest models landed within two runs of each other. Without one, results ranged from 7 of 15 to 0 of 15, and the order did not follow benchmark rankings: with no written rules in the repo, Sonnet 5.5 dropped a column the billing sync reads in 3 of 3 runs.

Written rules help, if they are read

In the Opus 5.5 runs, a 13-line API contract doc and three README lines stopped the breaking changes even with a one-line prompt. With the README in place, Opus 5.5 and Sonnet 5.5 kept the billing column in 6 of 6 runs; Haiku 4.5 skipped it and dropped the column in 3 of 3. Rules in the prompt reached every model.

What this does not show. Three runs per variant shows that something can happen, not how often. The repos are small and clean on purpose, and we wrote the specs and the test suites ourselves, so the specs cover what we test. The one gap a spec left open is reported on the coupon page rather than hidden.

Same prompt, three products: the coupon ticket

The vague prompt was one line: "Add coupon codes to checkout. Make sure invalid codes do not break payment." Two of three Opus 5.5 runs passed all 11 hidden checks, and every run passed its own tests. The difference was in decisions nobody asked for, which we measured by calling each run's code:

DecisionVague run 1Vague run 2Vague run 3Spec runs 1 to 3
How often can a user use a coupon?OnceUnlimitedOnceOnce
Coupon expires between apply and payCharges full price, $54.00Refuses the first paymentRefuses the first paymentCharges the discounted $43.20
How a bad code is reportedThrows an errorThrows an errorThrows an errorReturns { ok: false, reason }

Each vague run is defensible on its own. Together they are three different products from one ticket, and a reviewer reading one diff would not know the other two were possible. The spec runs matched each other, including on a case the spec itself forgot. For a longer walkthrough of the same pattern on a refund feature, see same feature, two approaches.

When to use vibe coding, and when to switch

SituationUse
Exploring options, UI copy, a prototype you will throw awayVibe coding
A small change in a repo whose written rules (an AGENTS.md or CLAUDE.md, README constraints) cover the risky partsA one-line prompt can be enough; check the rules really cover it
Anything touching money, permissions, user data, a schema, a public API or a shared helperSpec coding
Code that people who were not in the chat will maintainSpec coding

Explore loosely

Vague prompts are fine while you are comparing options, sketching copy or asking an agent to explain approaches. At this stage the output is material for thinking, not a decision to merge.

Freeze the decision before production code

Once you pick a direction, write the outcome, non-goals, acceptance criteria and evidence. The agent then implements the agreement instead of discovering product behavior through the diff.

Treat surprises as spec changes

If a run adds fields, states, dependencies or a cleaner architecture nobody asked for, do not merge the idea on the spot. Update the spec or write a follow-up, then run again.

Is spec coding just waterfall?

No. Waterfall specified a whole system months before anyone built it. A spec in spec coding covers one change, usually fits on a page, and is written after the exploring is done, often in under an hour. You can vibe-code the prototype and spec the production version of the same feature.

The criticism that holds is about bad specs: ones that dictate implementation instead of behavior, and ones nobody checks the code against, so they drift. Keep acceptance criteria about observable behavior, keep the spec in the pull request, and map each criterion to a test. Our technical spec guide shows the format.

Use vibe coding safely

  • Explore with vague prompts before production files are touched.
  • Convert the chosen direction into scope, non-goals, criteria and evidence.
  • Tell the agent which paths it may change before implementation.
  • Review extra behavior as drift, even when it looks useful.
  • Keep the final spec with the pull request.

Copy-ready vibe to spec checklist

Use this when a request sounds clear in conversation but still leaves the agent room to invent behavior.

vibe-to-spec.md
# Vibe Request To Spec

Vague request:
-

Clarifying questions:
- Who is affected?
- What exact behavior changes?
- What must not change?
- Which failure or permission path matters?
- What evidence proves it?

Spec version:
## Outcome
-

## Scope
- In:
- Out:

## Acceptance Criteria
- [ ] Given ..., when ..., then ...
- [ ] Failure path:
- [ ] Boundary path:

## Evidence
- Test:
- Manual check:
- Changed files:

Do not generate production code until this is testable.

Conversion example

The same request, rewritten so that it names behavior instead of a feeling.

filled-example.md
Vague:
- Make checkout feel more reliable.

Spec:
## Outcome
- Show a recoverable error when payment authorization times out.

## Scope
- In: payment submit handler, checkout error banner, payment tests
- Out: provider client API, database schema, order confirmation page

## Acceptance Criteria
- [ ] Given payment authorization times out, when the user submits checkout, then the page shows "Payment is taking longer than expected" and keeps the order pending.
- [ ] Given the user retries after timeout, when the provider succeeds, then the order moves to paid once.
- [ ] Given the provider returns card_declined, when checkout submits, then no retry banner is shown.

## Evidence
- npm test -- checkout-payment
- Screenshot: timeout banner state

Related resources

The recorded agent runs

Every prompt, diff and hidden-suite score behind the numbers on this page.

Open the runs

AI coding acceptance criteria

Turn the direction you chose into pass/fail behavior an agent can be checked against.

Write criteria

spec.md template

Commit the final decision next to the code as the agent's source of truth.

Copy spec.md

Vibe coding vs spec coding FAQ

What is the difference between vibe coding and spec coding?

Vibe coding gives an AI agent a plain-language prompt and judges the result by whether it runs and looks right, so the model makes the product decisions while it writes the code. Spec coding writes the outcome, non-goals, acceptance criteria and evidence down first, so people make those decisions and the agent builds against them.

Is vibe coding bad?

No. In our recorded runs, one-line prompts usually produced working code on small repos. The risk is that the same prompt can produce different products on different runs, and nobody reviews those decisions before they ship. It suits prototypes and exploration, not changes to money, permissions, data or shared APIs.

Is spec coding the same as spec-driven development?

Yes, in practice. Spec coding, spec-driven development and spec-first development all mean writing a short spec for a change before an AI agent implements it. Tools such as Spec Kit, OpenSpec and Kiro add structure around that idea; our tool comparison covers the main ones.

Does spec coding slow teams down?

The run itself barely changes: in our tests a spec run cost a few cents more than a one-line prompt and took somewhat longer (Opus 5.5 averaged 63 seconds instead of 47). The real cost is the time a person spends writing the spec, usually minutes to an hour for a typical change, which is paid back if it prevents one rework cycle.

When should I switch from vibe coding to spec coding?

Switch when the output will change production behavior, money, permissions, user data, a database schema, a public API or a shared helper, or when people who were not in the chat will maintain the code.

Explore freely, then write the spec before the generated code becomes a product decision.

Convert a prompt