A customer service AI assistant cost £140,000 to deploy and was signed off on schedule. It answered seventy per cent of enquiries without escalation, the team won an internal award, the deck went to the board. Fourteen months later the resolution rate had fallen to forty-one per cent, and the service desk had quietly gone back to answering by hand, because correcting the assistant took longer than writing the reply.
Nobody changed the model. Nobody changed the prompt. What changed was everything the assistant stood on: the price list went through two revisions, two products were withdrawn, the returns policy gained a new clause, and the CRM changed its ticket status vocabulary because the sales team asked for new categories. Every one of those was a correct business decision. None of them carried a step called “update the data the assistant runs on”, because no such step existed in any process — the project had ended, and with it the budget that could have obliged anyone to take it.
This article argues something that cuts across how this market prices AI: the largest cost of an AI deployment is neither the model nor the infrastructure but the maintenance of data quality over time — and it is that, rather than model accuracy on the day of sign-off, that decides whether the deployment survives its second year.
Why does the AI bill look different after two years than it did in the budget?
The standard way of budgeting AI splits the cost into phases: proof of concept, minimum viable product, production deployment. We do it too — our own breakdown of AI implementation cost from PoC to production is built exactly that way, and it is useful for what it describes: the effort up to go-live.
The problem is that this budget shape carries a silent assumption which, for AI, is false. The phase model assumes that after the final phase, cost drops to infrastructure maintenance — the way it does in a system implementation, where sign-off leaves you with a licence, hosting and support. In AI nothing drops after the final phase, because the principal cost driver does not live inside the system. It lives outside it, in the rate at which the world the data describes keeps changing.
A model learns a distribution. A distribution is a photograph of reality taken at a moment. Business reality then moves regardless of whether anyone tells the AI team: new products launch, prices change, regulations shift, contact channels are added, naming conventions and org structures are rewritten. Each change nudges the data a small step away from reality, and the sum of those steps has one property that has to be stated plainly: the degradation is silent. The system does not stop, does not throw an error, does not turn a dashboard red. It keeps answering — just wrongly more often, and in a tone that sounds confident.
That asymmetry is why the cost is so systematically missed. An infrastructure failure is visible in a minute and has an owner. Data degradation is visible over quarters and has no owner, because there is no event to report it. Budgets respond to events.
What is “data quality” actually made of, and why is it not a one-off cleanup?
“Improving data quality” sounds like a task at the front of a project: deduplicate, fill the gaps, normalise the formats. That is the one-off part, and usually the cheapest. The line item this article is about has six dimensions, five of which demand continuous work.
Completeness. The share of records with the fields the solution depends on actually populated. It degrades on its own whenever the process changes — a new form, a new optional field, a reordered sales flow.
Freshness. The distance between the state of the data and the state of the world. The only dimension that degrades even when nobody touches anything, purely through the passage of time.
Definitional consistency. Whether “active customer” means the same thing in the warehouse, in the CRM and in the head of the person who wrote the requirement. The most insidious dimension, because a definitional split leaves no trace in the data — only in the results.
Label correctness. For supervised solutions: whether the assigned categories still correspond to what those categories mean today. A product taxonomy change invalidates part of a labelled set immediately and silently.
Representativeness. Whether the data covers the cases that actually occur. Enter a new market or a new customer segment and the set stops representing what the system is being asked about.
Delivery continuity. Whether the source still delivers in the same shape, at the same time, with the same semantics. The most common cause of a sudden outage and the only dimension that usually has monitoring.
The first dimension can be handled by a project. The other five are a function of the rate of change in the business, not a function of how well the first deployment was executed. That is the difference between cleaning and maintaining — and equally the difference between a one-off line and a subscription.
What does the regulated world already say about this?
There is a useful reality check available to anyone arguing this line internally: in the domains where the cost of a wrong model output is highest, supervisors settled the question years ago, and they settled it against the phase model.
Model risk management guidance treats data quality as a component of model risk, not as a project task. The joint supervisory guidance issued by the US Federal Reserve and the OCC (SR 11-7 / OCC 2011-12) is explicit that a model’s output is only as sound as the data feeding it, and that data quality assessment belongs to the ongoing monitoring of a model in production — not to its development phase. Banks that adopted this framework did not do so because they are cautious; they did so because they were made to measure what everyone else discovers accidentally in year two.
Data protection law imposes an accuracy duty, not an accuracy preference. Under the UK GDPR and the EU GDPR alike, personal data must be accurate and, where necessary, kept up to date, with inaccurate data erased or rectified without delay. That converts part of your maintenance spend from a return-on-investment decision into an obligation — which matters practically, because obligations survive cost reviews and discretionary line items do not.
The observability vendor category exists because the problem is real and common. An entire product segment now sells detection of freshness, volume, schema and distribution anomalies in data pipelines. You may or may not want to buy such a tool. Either way, the existence of a funded market segment is evidence that the failure mode described here is neither rare nor exotic.
The practical read for a CTO briefing the board: you are not proposing an unusual precaution. You are proposing what regulated industries were required to do, applied to a use case that has not yet been required to do it.
Why does the model cost fall while the data cost rises?
These two curves are diverging, and their divergence is the main reason 2023’s cost proportions mislead today.
The model side falls for several independent reasons at once: inference gets cheaper with each hardware generation and each round of vendor competition, hosted models displace bespoke training for most enterprise use cases, and the tooling has matured to the point where work that took a quarter takes a week. That curve works in your favour and will keep doing so.
The data side rises because it is the product of two rising quantities. The first is the number of sources — a successful deployment immediately generates demand for further use cases, and each one connects new systems. The second is the rate of change inside those sources, which rises with the rate of change in the business. The product of two rising numbers rises faster than either.
The practical consequence: over a three-year horizon, the first year’s proportion misleads in both directions. In year one, model and build dominate the bill, because it is a build year. By year three, data maintenance dominates — and nobody recalculated the proportion in between, because nobody had a reason to. The project had formally ended.
From what we see in maintenance conversations, the annual data quality cost for a production solution typically lands somewhere between the low tens of per cent of the original implementation cost, with the upper part of that range applying to solutions built on many sources and on data describing a fast-moving product offering. That is an order of magnitude rather than a rate — but even as an order of magnitude it is enough to change the scope decision on a first deployment.
Why is year two when deployments die?
Year two is not arbitrary and not industry superstition. Four clocks converge in it, and all four start ticking on the day of sign-off.
Clock one: drift crosses the threshold of noticeability. Individual changes to offering, process and vocabulary do not move the result visibly. Their sum after a dozen months does. Degradation is gradual right up to the point where it crosses the line users were still tolerating.
Clock two: the project budget closed. The contingency that funded fixes in the first months disappeared with the project reconciliation. Maintenance carried forward into the operating budget at a level that covers infrastructure, because that is what the business case described.
Clock three: the person who understood the data left. Knowledge of why one field is trustworthy and another must be ignored rarely reaches documentation. Turnover in data teams means that after a year or two nobody can reconstruct the intent behind a transformation.
Clock four: user trust runs out non-linearly. People tolerate tool errors up to a threshold, and past it they abandon the tool in a step, not a slope. Rebuilding trust costs several times what maintaining it would have, because it requires not just a fix but a demonstrated fix.
The conclusion is uncomfortable for both sides of a commercial conversation: an AI deployment planned without a maintenance line is not a cheaper deployment. It is a deployment with a shorter horizon, where the difference is paid later and at a worse unit price — because restarting always costs more than continuity.
How do you price annual data maintenance before signing the project?
The calculation needs no tooling and no consultant, and fits inside a week. Three quantities, all historically measurable.
The number of independent sources. Count the systems the solution genuinely draws from — not intermediate layers, but the places where the data is created. Each source has a separate owner, a separate change schedule and a separate failure mode. Maintenance cost scales with this count almost linearly, and coordination complexity faster than linearly.
The historical rate of schema change. For each source, check the change history of the last two years and count how many times the structure or the definition of a field the solution depends on actually changed. This is the only forecast of future change that is not guesswork, because the previous two years are the best available approximation of the next two.
The share of data requiring human judgement. Only that share generates a relabelling cost, and only it needs a budget for people rather than machines. Its size determines whether the maintenance line is moderate or dominant.
Two smaller fixed lines sit alongside these and are easier to estimate: data quality monitoring (automated freshness, completeness and distribution tests) and the time spent responding to alerts, which is a real fraction of an FTE rather than zero. Collect these in the same spreadsheet where you calculate implementation cost from PoC to production — as a separate line repeating annually, not as a percentage uplift.
What does relabelling cost, and why does nobody budget for it?
Labelling is the one AI cost that is highly visible at the start and invisible afterwards — and that pairing explains why it falls out of the calculation.
At the start it is impossible to miss: someone has to describe the set, it is human work, it has a price and a deadline, so it enters the budget without argument. Our guide to data annotation as a strategic process covers that phase in detail.
It becomes invisible the moment the set is finished, because intuition says a label is permanent. It is not. A label is a claim about the world, and it expires with the world. A product taxonomy change invalidates part of a labelled set immediately. A new support ticket category means the existing set contains not one example of a class that will occur from tomorrow. A policy change moves the boundary between two classes, making a portion of historical labels simply wrong.
A practical rule worth adopting in planning: in domains where the offering or the process changes several times a year, an annual refresh touches a meaningful fraction of the original set — not a couple of per cent. Organisations that do not budget for it pay the cost anyway, in a worse form: as an accuracy decline that only becomes visible with a lag and carries an additional bill in lost user trust.
There is also a straightforward move that lowers this line, worth considering before the first labelling round: designing the taxonomy to be robust to predictable changes in the offering. Labels describing customer intent age far more slowly than labels describing a product name, and the difference in maintenance cost between those two designs is a multiple, not a margin.
Does a hosted model or a RAG layer remove this cost?
This is the strongest objection to the article’s thesis and it deserves an honest answer, because within a certain range it is correct.
It is correct in this: choosing a hosted model over training your own genuinely removes large lines from the bill — the training set, the compute, the research team. The build versus buy decision for AI solutions really does shift a large share of cost to the vendor, and for most enterprise use cases it is today the right call.
It is incorrect in suggesting that this removes the line this article is about. It relocates and renames it. In a setup that retrieves from your own documents, answer quality is a function of retrieval quality, and retrieval quality is a function of the quality and freshness of what was indexed. A vector store built on last year’s procedures returns last year’s procedures, and it does so with a linguistic fluency that is itself an additional hazard: the answer sounds credible regardless of whether the source is current.
What does change are the specific lines in the bill, and naming them changes the plan:
- training and training-set labelling costs disappear;
- the freshness cost remains and grows — who checks how often that indexed documents are current, and who retires the ones that are not;
- a contradiction resolution cost appears — when two documents disagree, someone must decide which one governs, because the system will not;
- the definitional consistency cost remains exactly as it was under supervised learning, because it is an organisational problem rather than a technical one.
The practitioner’s takeaway: architecture changes the structure of the data maintenance cost and can reduce it, but it does not reduce it to zero in any scenario where the data describes something that changes.
When does this thesis not apply?
There are cases where the data maintenance cost really is close to zero, and honesty requires naming them — a thesis stated without exceptions is a slogan rather than an analysis.
Closed, static data. Recognising content in documents with a stable, multi-year format; processing scans against a fixed template; classifying objects whose definition does not depend on business decisions. The world those data describe simply does not move, so there is nothing to maintain.
Data that is the system. An assistant working over a code repository has data kept current by the team’s ordinary work — freshness is a by-product of the process rather than a separate task. That is the healthiest possible arrangement and worth looking for when choosing a first use case.
One-off analytical work. A single analysis of a historical set to support one decision needs no maintenance, because it has no horizon over which it could age.
There is a genuinely useful conclusion here for selecting a first deployment: use cases differ in maintenance cost by an order of magnitude, and that difference is visible before the project starts. Choosing between two candidates with comparable returns, take the one whose data changes more slowly — you are buying not only a lower cost but a longer service life.
Who should own this budget line?
Where the cost sits matters more than how large it is, because it decides whether anyone holds both the knowledge and the leverage.
Not the AI team. The team sees the symptom — falling accuracy — but has no influence over the cause. It does not decide that sales staff will stop filling in a field, or that a source system will change its vocabulary. Keeping the line inside the AI team’s budget means it gets cut at the first cost review as “tool maintenance”.
Not IT. IT is accountable for data flowing, and generally does that well. It is not accountable for whether the data means what it is supposed to mean, because that is a question about a business process.
The owner of the process where the data is created. This is the only role with both knowledge of what the data means and leverage over how it is produced. A sales director accountable for customer data quality has instruments to act on it; an AI team lead has an alert.
The practical way to seat that accountability without inventing a new function is a data contract between the source owner and the consumer. It has four elements and fits on two pages: a description of the structure and meaning of the exposed fields; a freshness commitment, meaning the maximum lag with which the data reflects reality; a commitment to give notice of breaking changes, with a specific notice period; and a named owner on each side. Data contracts are a standard practice in decentralised architectures — the data mesh approach describes them as part of the model — but their value does not depend on that architecture and needs no transformation programme to begin.
What does the data quality maturity map look like?
The map below lets you place your own organisation in a few minutes. The symptom column matters more than the level name, because the symptom is how you recognise where you are.
| Level | Who owns data quality | How degradation is detected | Budget position | Typical symptom |
|---|---|---|---|---|
| 0 — none | nobody | by user complaint | none | the deployment quietly stops being used and nobody files an incident |
| 1 — reactive | the AI team, after the fact | by an accuracy drop noticed at a review | ad hoc, from contingency | a firefighting cycle every few months, always after the fact |
| 2 — monitored | the data team | automated freshness and completeness tests | a standing maintenance line | alerts exist, but nobody has the mandate to force a fix at the source |
| 3 — contractual | the source owner, under a data contract | tests plus a breaking-change notice commitment | a line owned by the process owner | schema changes arrive as notice, not as an outage |
| 4 — embedded | the business process | quality measured where the data is created | a process cost, not an AI cost | new use cases start in weeks rather than quarters |
The step that changes the economics is between level 2 and level 3, and it is not a technical step — at level 2 the organisation already knows everything; it simply has nobody with the mandate. Level 3 requires one management decision: moving the budget line and the accountability to the source owner. It requires no new tool.
What are the key takeaways?
AI deployments rarely die because the model was bad. They die because the data they stood on drifted away from reality while no budget line and no named person existed to prevent it. The phase model — PoC, MVP, production — describes only the cost of building, and quietly implies that sign-off leaves you with infrastructure maintenance. In AI, sign-off is where the line that decides the outcome begins.
Three things are worth doing before signing the next deployment. First, count the sources and check the change history for how many times each one’s structure moved in the last two years — the best available forecast of the next two. Second, add a separate, annually repeating line for data quality maintenance instead of hiding it inside a maintenance uplift. Third, name an owner for that line on the business side rather than in the AI team — and if no name can be produced, treat that as information about the organisation’s readiness rather than a detail to settle later.
When choosing the first use case, weigh the rate of data change alongside the expected return, with the same discipline you would apply to an AI vendor evaluation. A lower-return deployment on slowly ageing data is often a better first decision than a spectacular one that will age out within a year.
At ARDURA Consulting we build AI and data engineering solutions and we open every deployment conversation with a question this industry asks too rarely: after sign-off, who is accountable for the data continuing to describe reality? Let’s talk about your use case before year two prices it for you.