Skip to content
SwankyForge
SwankyForge
All insights
Article9 min read

A vibe-coded prototype costs a weekend. Production costs the rest.

AI made prototypes nearly free, but the work that turns a demo into a system people rely on did not shrink, so a prototype should be budgeted as a specification rather than as most of the product.

In November 2025 Collins Dictionary named "vibe coding" its word of the year and defined it as using AI, prompted in natural language, to help write code. The phrase stuck because the experience is real. Describe an app, accept what the model writes, and there is a working screen by Sunday evening. A founder can now show a demo to customers before hiring an engineer.

What did not get cheaper is the part that makes software dependable. In the 2025 Stack Overflow Developer Survey, 66% of developers named "AI solutions that are almost right, but not quite" as their top frustration with AI tools, and 45% said debugging AI-generated code takes more time. A prototype is the place where almost right is fine. Production is the place where it is not.

This note lists what production adds to a prototype, gives a rough split of the effort, and suggests how a budget owner should treat the demo they were just shown.

The speed-up is real, and still unmeasured

The most careful measurement of AI coding speed comes from METR, a research nonprofit. Its 2025 randomized trial found that experienced open-source developers took 19% longer on tasks when AI was allowed, with a confidence interval of +2% to +39%.

On 24 February 2026 METR published a follow-up: 57 developers, 143 repositories, more than 800 tasks, run from August 2025. On the same time-change scale where the 2025 result was +19%, the new point estimates are -18% for developers returning from the first study (interval -38% to +9%) and -4% for newly recruited developers (interval -15% to +9%). Both point estimates now lean towards AI saving time. Both intervals include zero.

METR itself calls the new data an unreliable signal. Between 30% and 50% of developers said they had chosen not to submit some tasks because they did not want to do them without AI. The tasks where AI helps most are therefore missing from the sample, and METR believes the true speed-up is likely larger than its estimate.

For a buyer, the conclusion is narrow. AI makes some coding faster, by an amount that careful researchers have not been able to pin down. It does not change what a system needs before people can depend on it.

What a prototype proves, and what it does not

A weekend prototype proves three useful things: the idea works on the inputs someone tried, the interface makes sense to the people who saw it, and somebody wants it.

It does not prove that the system works on inputs nobody tried, that it works for two hundred people at once, that it shows each person only what they are allowed to see, that it costs what you think per month, or that anyone can fix it when it breaks at night. Those are the questions production answers.

There is a second difference that is easy to miss. A prototype is judged by the person who built it, on the day it was built, with inputs they chose. A production system is judged by people who never saw the demo, months later, on data that did not exist when the code was written. Most of the work in the next table exists so that the second judgement goes well.

What production adds

The shares below are an illustrative split for an internal LLM tool, such as a document-search copilot, going from a working prototype to a few hundred daily users. They are planning opinions, not measurements. Change the weights for your own case; the shape is the point.

Work item What it means Without it Illustrative share
Core logic What the prototype already shows No product 20%
Data contracts and pipelines Schema checks on every input feed; scheduled, idempotent loads An upstream export changes and answers degrade without an error 15%
Evaluation set and quality gates Real questions with expected answers, run on every prompt or model change Nobody can tell whether a change made things better or worse 12%
Authentication and permissions Single sign-on; access rules enforced at retrieval time Staff see documents they should not 8%
Input validation and error handling Size and format limits; handling of hostile text in uploads and retrieved documents One odd file takes the service down or steers the model 8%
Monitoring and alerting Latency, error rate, cost per request, empty-retrieval rate, user feedback Problems are reported by users, days later 8%
Security review and fixes Secrets, dependencies, data sent to third-party APIs A leaked key or a data-handling breach 8%
On-call and runbooks A named owner and written steps for the common failures The one person who knows is on holiday 7%
Rollback and release process Versioned prompts, models and indexes; a switch back to the last good version A bad release stays live while someone debugs it 5%
Documentation and handoff How it works, how to change it, how to run it The system cannot outlive its author 5%
Cost limits Per-user quotas, token caps, a spend circuit breaker A loop or a heavy user produces a surprise invoice 4%

The first row is the prototype. In this split it is a fifth of the work. The other four fifths are invisible in a demo, which is why a demo is such a poor guide to a budget.

For engineers: where the time goes

A few of the rows deserve more detail, because they are where estimates usually go wrong.

Permissions belong in retrieval, not in the answer. In a retrieval-augmented system, the access check has to filter documents before they reach the model. Filtering after generation does not work: the model has already read the restricted text and may paraphrase it. That means the index carries access metadata, it stays in sync with the source system, and someone tests it with accounts that have different rights.

The evaluation set is the specification in executable form. It is a versioned file of real inputs with expected outputs or grading rules, run in CI on every change to prompts, models, chunking or retrieval settings. Building it means sitting with the people who will use the system and writing down what a correct answer is. That is slow, and no assistant can do it for you, because the knowledge sits with the users.

Input validation now includes text that attacks the model. A document in the index can contain instructions. Uploads can be huge, empty, scanned images or the wrong encoding. Each case needs a defined behaviour and a test.

Rollback needs everything versioned together. A release is a prompt, a model version, an index build and code. If only the code is in git, rolling back the code does not roll back the behaviour.

Cost limits are a production feature. A prototype has one user. A production system has a few hundred, plus scripts and retries. Per-user quotas, maximum token counts and an alert on daily spend are cheap to build and prevent the most common unpleasant surprise.

How to treat a prototype if you own the budget

Treat it as a specification. A working prototype is the clearest statement of what you want that most projects ever get. Hand it to the engineers together with the list of inputs it was tried on and the cases where it failed.

Do not treat it as 80% of the product. If a quote is small because the prototype exists, ask which of the rows above it covers. If the answer is only the first row, the quote is for a second prototype.

Use it to test demand before you fund production. Put it in front of the intended users for two weeks. If they do not come back to it, you have saved the production budget, which is the most useful thing a prototype can do.

When to throw the prototype away

Rewrite rather than extend when one or more of these is true:

  • secrets are in the code or in the repository history;
  • data is reached through somebody's personal account or personal API key;
  • the interface, the logic and the data access are mixed together, so permissions cannot be enforced in one place;
  • nobody can explain how a particular answer was produced;
  • it runs on a hosted builder that cannot be deployed inside your environment or meet your data rules;
  • the logic is long and has no tests.

Keep it when the core logic is short, readable and separable, a few tests already exist, and the data flow resembles the production one.

Throwing code away costs less than it sounds. The prototype has already done its job: the decisions are made, the examples are collected, the interface has been tried. The prompts, the example inputs and the screenshots usually survive a rewrite even when the code does not.

What the gap looks like in money

Our own published price bands show the same ratio. A proof of concept is €14k–38k over 4–7 weeks. A production build is €45k–120k over 10–20 weeks. At both ends of the range, production is about three times the proof of concept: 45 / 14 ≈ 3.2 and 120 / 38 ≈ 3.2. That multiple is the cost of the rows above that are not the first one. We ship monitoring and rollback before handoff for the same reason: they are what lets a client run the system without us.

If you already have a prototype and want to know which rows it covers, the free teardown is a first check in six questions, and a short paid discovery answers it in full. The same list applies to internal document copilots, where permissions and the evaluation set usually take the most time.

Sources

Have a process that costs too much?

Tell us about it. The first call is free, and if AI is not the answer we say so.

Discuss your task