The AI Integration Invoice: What Six Retrofits Actually Cost, Line by Line

Where the hours actually go in an AI retrofit

Across the six retrofits we shipped, roughly a third of engineering hours went to discovery and data plumbing, a fifth to prompt and model work, and the rest to evals, review UI, fallbacks, and integration glue. The model call itself was never the largest line on the invoice.

The shape is consistent enough that we now quote from it. A retrofit into a product that is already in production starts with a data audit, because the feature is only as good as the fields you can legally and reliably read at call time. On GlycanAge, that audit was the single largest discovery item: working out which parts of a customer record could cross a provider boundary under the consent captured at signup, and which had to be redacted or replaced with derived values before the prompt was assembled. On Plodovi, discovery was cheaper because product data is not personal data, but the catalogue had four generations of inconsistent attribute naming that had to be normalised before any copy generation was worth running.

The pattern founders find surprising is how little time goes into prompting. Prompt iteration is fast, cheap, and mostly done by a product person once the harness exists. What is expensive is everything that makes the prompt safe to run unattended: the eval set, the human review queue, the retry and fallback path, the logging, and the cost ceiling.

• • Discovery and data audit: about 30% of hours, higher on anything touching personal data
• • Prompt, model selection, and output schema design: about 20%
• • Eval harness and test set curation: about 15%
• • Human review UI and moderation queue: about 15%
• • Fallback path, retries, caching, and cost guards: about 10%
• • Integration glue, deploys, monitoring, docs: about 10%

Cost per 1,000 calls at production volume, per feature

Per-call cost varied by two orders of magnitude across the six features, driven almost entirely by input size and whether the output could be cached. Short, structured features like support triage cost fractions of a cent per call. Long-context features that assemble a customer record cost dramatically more, and are the ones worth optimising first.

The three levers that moved cost were, in order of impact: trimming the input, caching at the right key, and only then switching models. Trimming input meant replacing a whole record dump with a hand-picked field set, which on the health-data feature cut prompt tokens by more than half and had the side benefit of reducing what left our infrastructure at all. Caching mattered most on Plodovi, where product copy is generated once per product version and read thousands of times. Once we keyed the cache on a hash of the normalised attribute set rather than the product ID, regeneration only happened when attributes actually changed, and monthly spend on that feature dropped to a fraction of the launch figure.

Model switching was the last lever, and the smallest. Moving a feature from a frontier model to a smaller one cut the per-call price meaningfully, but only after the eval suite existed to prove output quality had not degraded. Without evals, a model switch is a guess with a cost saving attached, and the two times we watched teams do it without evals, they switched back within a month. The full per-feature mechanics, including latency budgets and where each call sits in the request path, are in our retrofit log.

The three cost lines founders never budget for

Founders budget for the model call and the engineering time to make it. They almost never budget for the eval harness, the human review interface, or the fallback path, which together consistently account for 35 to 40 percent of an AI retrofit's engineering hours. These are not optional extras. They are what separates a demo from a feature you can leave running.

The eval harness is the one people push back on hardest, because it produces nothing a user sees. It is also the only thing that lets you change a prompt or a model without a week of manual spot checks. Ours are deliberately unglamorous: a set of 40 to 150 real inputs with expected properties of the output, run as a script, with a pass rate printed to the console. Not a platform, not a dashboard. But without it, every prompt change is a production experiment.

The human review UI is the second. Any feature that writes customer-visible text or influences a clinical or financial decision needs a queue where a person approves, edits, or rejects before publication, at least for the first months. That is a real interface with real states, and it costs real hours. The fallback path is the third: what the product does when the provider is slow, rate limited, or returns something that fails schema validation. On every feature we shipped, the fallback is the pre-AI behaviour, preserved rather than deleted. That decision costs a little up front and saves an incident later.

• • Eval harness: a versioned set of real inputs plus assertions, run on every prompt or model change
• • Human review UI: approve, edit, reject states plus audit of who approved what
• • Fallback path: the previous non-AI behaviour, kept alive and exercised, not a placeholder error

Glue code versus model code: the ratio nobody quotes

In every retrofit we have done, the code that actually calls the model is a small fraction of the code shipped, typically well under a tenth. The rest is input assembly, redaction, schema validation, retry logic, caching, logging, cost accounting, queueing, and the admin surfaces that let someone see what the feature did and why.

This ratio is why AI integration into an existing product is priced like integration work, not like research. The hard parts are the same hard parts as any other feature that touches multiple systems: where does the data come from, what is the contract, what happens on failure, who can see the result. A team that has shipped payment integrations or third-party APIs into a live product already has most of the skills. What they lack is the habit of treating a non-deterministic dependency as a normal one, with schema validation on the way out and a defined behaviour when the contract is broken.

The practical consequence for scoping: ask an agency how they validate model output, what happens on a schema failure, and where the cost ceiling lives. If the answer is only about prompts and model choice, the estimate is missing 80 percent of the work.

The two engagements where we mispriced scope and wrote off time

Two of the six retrofits were quoted wrong and we absorbed the difference. In both cases the cause was the same: we priced the AI feature and not the data work underneath it. One was a content generation feature where the source data was far dirtier than the sample we scoped from. The other was a triage feature where the routing rules we were asked to automate did not actually exist in writing.

In the first, we scoped copy generation off a sample of a few hundred well-formed catalogue records. In production, a meaningful share of records had attributes in free-text notes fields, inconsistent units, and supplier-specific abbreviations. The generation code worked. The normalisation work required to make its inputs trustworthy was effectively a second project, and it was not in the quote. We finished it and did not bill the overage, because the sampling error was ours: we should have pulled a random slice of the real table, not the one the client showed us.

In the second, the brief was to auto-route inbound support messages using existing categories. The categories existed in a dropdown. The routing logic lived in one person's head and contradicted the dropdown in several places. Every eval round surfaced another undocumented rule, and we were three rounds deep before we admitted that what we were being paid to automate first had to be written down. We now treat that as a precondition: if the rule set cannot be produced as a document before work starts, the discovery phase is priced separately and the implementation quote waits. Both write-offs changed how we scope, which is cheaper than repeating them.

How to read an AI integration quote

A credible AI integration quote separates discovery from implementation, names the eval and fallback work as its own line, states a monthly inference cost estimate with the assumed call volume, and says what happens if the data turns out worse than the sample. A single blended number for "AI feature" hides every risk that actually bites.

The volume assumption matters more than the unit price. A per-call cost is meaningless without a stated calls-per-month figure and what happens above it, so we quote a ceiling and an alerting threshold rather than an open-ended API key. It is also worth asking who owns the prompt after handover, who runs the evals, and who watches the bill, because those are ongoing responsibilities that do not end when the feature ships.

If you are evaluating agencies for a retrofit into a product that is already live, the useful question is not whether they have used an LLM API. It is whether they can show you the failure behaviour, the review queue, and the cost line from something running in production. That is the substance we try to publish, feature by feature, rather than describe in the abstract.

The feature-by-feature mechanics behind these numbers, including where each call sits in the request path and the two features we turned off, are in our AI retrofit log.