The Feature That Wasn't Worth an LLM: 9 AI Requests We Solved With SQL, a Rules Table, or Search

Why the triage step comes before the integration

Most AI integration requests on an existing product are actually requests for a capability, not for a model. Before quoting an LLM, we run a four-question triage: is the output set bounded, is the ground truth already in the database, does a wrong answer cost money, and does the team have anyone to own prompt drift. Three yeses usually mean no model.

The reason this matters more for teams without ML staff is ownership, not accuracy. An LLM feature is not a build, it is a subscription to a behaviour that changes when you change models, when the provider updates the model silently, or when your own data distribution shifts. A rules table and a SQL view do not drift. They break loudly, in a way a backend developer who has never read an ML paper can diagnose from a stack trace.

So the triage question is never "could a model do this". A model can do almost all of it. The question is whether the marginal quality over a deterministic version is worth adding a non-deterministic dependency to a product that a small team has to keep running for years.

• • Bounded output set: if the answer is always one of N known values, you want a classifier or a lookup, and most of the time a lookup.
• • Ground truth already stored: if the answer is derivable from rows you already have, an LLM is guessing at something you can compute.
• • Cost of a wrong answer: a wrong route suggestion wastes fuel, a wrong result explanation in regulated health tech is a compliance event.
• • Owner availability: if nobody on the team can own evals and prompt changes, the feature has no maintainer from day one.

The nine requests, and what shipped instead

Nine AI requests arrived across client work on products that were already live. Seven shipped as deterministic implementations: SQL views, a rules table, trigram search, a scored heuristic, or a template system. Two were correctly LLM-shaped from the start. The pattern in the seven is that the request described an output format, not a reasoning problem.

Result explanations in a diagnostics context is the clearest example. The ask was "use AI to explain the biomarker result to the customer in plain language". The actual requirement was a clinically approved explanation per result band. That is a rules table keyed on biomarker, band, age bracket, and sex, with copy written once by a clinician and reviewed on a schedule. An LLM paraphrasing approved medical copy adds a hallucination surface to a sentence that must be exact. The deterministic version is auditable, versioned, and cheaper to review than a prompt.

Duplicate supplier detection is the second clearest. The ask was an embedding-based similarity service. What shipped was Postgres trigram similarity on normalized names plus an exact match on tax ID and a distance check on coordinates, surfaced as a merge suggestion queue for an admin. Normalization did most of the work: stripping legal suffixes, collapsing whitespace, folding diacritics. Embeddings would have found the same pairs plus a longer tail of false positives, at the cost of a vector store nobody was staffed to operate.

• • Result explanations for test outputs to a rules table keyed on biomarker, result band, and demographic bracket, with clinician-authored copy.
• • Duplicate supplier detection to trigram similarity on normalized names plus tax ID exact match plus geo distance, queued for admin merge.
• • Support ticket triage to keyword and route rules over ticket subject, order state, and account type; unmatched tickets go to a default queue.
• • Demand forecasting for perishables to SQL over trailing weeks with day-of-week and seasonality coefficients, exposed in the BI dashboard.
• • Driver route suggestions to constraint-based batching on delivery window, zone, and vehicle capacity; no learning component.
• • Product copy generation for catalogue imports to a template system with attribute slots, plus an editorial queue for the top-selling SKUs.
• • "Smart" search on a marketplace to a synonym dictionary, stemming, and weighted fields in the existing search index.
• • Free-text field normalization at intake to an LLM, correctly, because the input space was genuinely unbounded.
• • Long-form editorial summarization to an LLM, correctly, because the output was novel prose and a human reviewed every one.

The measured delta: what deterministic actually bought

Across the seven deterministic builds, the consistent wins were latency, cost, and explainability, not accuracy. Accuracy was roughly comparable on the common cases and worse on the long tail. Latency went from network-dependent hundreds of milliseconds to a single indexed query. Per-call cost went to zero. Explainability went from "the model said so" to a row an admin can read.

Forecasting is where the accuracy comparison is most honest. A trailing-average SQL model with day-of-week and seasonality coefficients is not sophisticated, and it is wrong in ways you can predict: it lags step changes, and it cannot see a heatwave or a public holiday you did not encode. But the operational question was never "what is the optimal forecast". It was "how much should be harvested on Thursday". A number an operator can sanity-check, adjust, and override beats a better number they do not trust. That is not an argument against models in forecasting. It is an argument for shipping the version people will actually use first, and measuring the residual error before buying anything.

The hidden win is review cost. A rules table gets reviewed in a spreadsheet by the person who owns the domain. A prompt gets reviewed by whoever wrote it, against an eval set somebody has to build and maintain. For a team of three, the second option consumes a role that does not exist.

The two that failed: where we retrofitted a model six months later

Two deterministic builds failed under real traffic and were replaced with LLM calls roughly six months in. Support triage failed on rule sprawl. Catalogue product copy failed on coverage. In both cases the failure signal was measurable before the rewrite, which is the point: deterministic first is cheap to abandon.

Support triage started as about a dozen routing rules over subject keywords, order state, and account type. It worked. Then it grew, because every misroute produced a request for one more rule, and rules written by different people started contradicting each other. The tell was maintenance frequency: when the rules table changes more often than the code around it and nobody can predict the effect of a new row, the abstraction is wrong. The replacement was a classification call constrained to the existing queue list, with the old rules retained as a pre-filter for the unambiguous cases and as the fallback when the model call fails or returns anything outside the allowed set.

Product copy failed differently. The template system produced acceptable copy for products with complete attributes and nothing usable for the long tail of imports with three fields filled in. Editorial covered the top sellers and the rest shipped with a stub. The coverage gap was the business problem, and templates could not close it because the missing input was the input. The LLM version generates a draft from whatever attributes exist, writes to a draft state rather than the live catalogue, and a human publishes. Note what survived: the templates still handle the well-structured products, because they are free and deterministic. The model handles the tail.

The rubric you can run on your own backlog

Run each AI request through six checks in order and stop at the first one that resolves it. Most requests resolve in the first three. The checks are: is the output bounded, is the answer computable from stored data, is it a retrieval problem, is it a formatting problem, is the input space unbounded, and does a human review every output. The last two are the genuine LLM signals.

The order matters because it front-loads the cheap answers. "Bounded output" catches classification requests that are really lookups. "Computable from stored data" catches analytics requests dressed up as prediction. "Retrieval problem" catches the very common case where the ask is semantic search and the fix is a synonym dictionary and correct field weighting in the index you already run. Only after those three fail should you be pricing tokens.

Two more rules we apply regardless of outcome. First, build the deterministic version even when you are confident the model will win, because it becomes your fallback and your eval baseline: if the model cannot beat trigram matching or a template, you have learned that for the cost of a day. Second, write down the failure signal in advance. "We revisit this if the rules table exceeds 40 rows" or "if coverage stays below 60 percent after editorial" turns an architectural argument into a measurement, and gives whoever inherits the system permission to change it.

• • Bounded output set to a lookup table or rules table.
• • Answer derivable from existing rows to a SQL view or scheduled aggregate.
• • Semantic matching over known records to normalization, trigram or full-text search, a synonym dictionary, and field weighting.
• • Structured input, predictable output shape to a template with slots.
• • Genuinely unbounded input to a model call, constrained output, deterministic fallback.
• • Novel prose with a human reviewer in the loop to a model call, draft state, explicit publish step.

If a request does survive triage and a model is genuinely the right call, the next question is who owns its behaviour, which we cover in the four ownership roles we assign instead of an ML hire.