After it's built, an AI system keeps costing money in eight places: model usage, hosting and infrastructure, third-party tools, monitoring, evaluation upkeep, prompt and knowledge maintenance, model upgrades, and the human time spent reviewing its work. Over a few years, these running costs can match or exceed the original build cost, which is why AI total cost of ownership (TCO) is the number to plan around, not the development quote. The good news is that most of these costs are predictable once you know to look for them, and several can be controlled by design.

This post breaks down each cost line, explains what drives it up or down, shows where businesses get surprised, and ends with a TCO worksheet you can fill in for any AI project, whether you build it, buy it, or do something in between.

Why is the build cost only part of the picture?

Traditional software has running costs too, but they're relatively stable: hosting, support, the occasional bug fix. AI systems behave differently in three ways.

First, usage costs scale with activity. Every conversation, document or task processed calls a model, and you pay per call. Double the volume and that line roughly doubles.

Second, the ground moves under them. Model providers update and retire models, your products and policies change, and customers find new ways to ask things. An AI system that isn't maintained slowly gets worse even if nobody touches the code.

Third, they need supervision. Somebody reviews flagged items, checks a sample of outputs, and handles escalations. That time is real cost, even if it never appears on an invoice.

If your build quote is the only number in the business case, the business case is incomplete. Our guide on how much it costs to build an AI agent covers the upfront side; this post is about everything after.

What are the running costs of an AI system?

Model usage

You pay the model provider per token (roughly, per chunk of text in and out), or you pay for the hardware to run an open model yourself. The main drivers:

  • Volume: how many tasks per day.
  • Context size: how much text goes into each call. Agents that stuff long documents, whole conversation histories or many retrieved passages into every request cost far more per task.
  • Steps per task: a simple classifier makes one call; an agent that plans, calls tools and checks its work may make ten.
  • Model choice: larger models cost more per token. Many tasks run fine on smaller, cheaper ones.

This is the line most likely to surprise you, usually because a prototype tested on twenty examples behaves differently at thousands per day. Our post on why AI bills grow so fast covers the levers in detail.

Hosting and infrastructure

Servers or serverless functions for your agent, a database, a vector database if you use retrieval, file storage, queues for background jobs, and logs. For most SMB systems this is modest compared with model usage, but it grows with data retention. Keeping every conversation and every log line forever is a cost decision, as well as a privacy one.

Third-party tools and integrations

Voice platforms, telephony, WhatsApp Business messaging charges, OCR or document-parsing services, automation platforms, and the API tiers of systems your agent connects to. Some CRMs and ERPs charge for API access or limit calls by plan, which can force an upgrade you didn't budget for.

Monitoring and observability

You need to see what the AI is doing: traces of each task, errors, latency, cost per task, and alerts when something drifts. That might be a tool subscription, some engineering time, or both. Skipping it doesn't save money; it just means you find problems from customer complaints. See how to monitor AI agents once they're live.

Evaluation upkeep

A good AI system has a test set: real examples with known correct outcomes that you run before every change. That set needs maintaining as your business changes. New product lines, new document formats and new failure cases all need adding. Running the tests also costs model usage.

Prompt and knowledge maintenance

Prices change. Policies change. A new service launches. The knowledge base, the instructions and the examples the AI relies on have to keep up, or it confidently tells customers last quarter's return policy. Someone has to own this, and it's usually a few hours a month for a small system, more for a large one.

Model upgrades and vendor changes

Providers release new models and retire old ones on their own schedules. Moving to a new model can improve quality or reduce cost, but it can also change behavior in ways that break your workflows, so each migration needs testing. We cover this in detail in what happens to your AI system when the model changes. Vendor pricing and terms can change too, which is one reason to avoid hard-wiring your system to one provider where you don't need to.

Human review and exception handling

The cost most often left out. If the AI handles 70% of invoices end to end and routes 30% to a person, that 30% is still your team's time. Add the time spent spot-checking automated work, answering escalations, and correcting the occasional mistake. This number should shrink as the system improves, but it rarely reaches zero, and for high-stakes work it shouldn't.

Which AI costs surprise businesses most?

From the builder's side, the same surprises come up again and again:

  • Context creep. Each new feature adds "just a bit more context" to every prompt. A year later, every call carries several times the text it started with.
  • Agent loops. An agent that retries, re-plans or calls tools in circles on hard cases can spend far more on those few cases than on all the easy ones combined.
  • Usage limits nobody set. A public chatbot gets hammered by bots or a single curious user, and nothing caps it.
  • The forgotten owner. Nobody is assigned to update the knowledge base, so quality decays, so staff stop trusting it, so the investment quietly fails. That's a cost too, even if it never shows up as a line item.
  • Deprecation deadlines. A provider retires a model, and a migration that should have been a planned project becomes an emergency.

How do you keep AI total cost of ownership under control?

Most of these costs can be designed down:

  1. Right-size the model per step. Use a smaller model for routing, classification and extraction; save the large model for the steps that need reasoning.
  2. Trim context. Send only what the task needs. Summarize long histories instead of resending them.
  3. Cap and cache. Set per-user and per-day limits, maximum output lengths, and reuse results for repeated requests.
  4. Cap agent effort. Limit steps and retries per task, and escalate to a person instead of looping.
  5. Measure cost per outcome, not per call: cost per resolved ticket, per processed invoice, per qualified lead. That's the number that tells you whether the system is worth it.
  6. Keep a model-agnostic layer so switching providers or models is a tested configuration change rather than a rebuild.
  7. Assign an owner with a few hours a month for maintenance, and put it in their job description.

A TCO worksheet for any AI project

Fill this in for a three-year horizon. Use rough estimates; the point is to make every line visible before you commit. For each line, note who owns it.

Cost lineWhat to estimateYear 1Year 2Year 3
Build or implementationDevelopment, integration, testing, data preparation
Model usageTasks per month x average cost per task (include agent steps)
Hosting and infrastructureServers or functions, databases, vector store, storage, logs
Third-party toolsVoice, messaging, OCR, automation platforms, API tier upgrades
Monitoring and observabilityTool subscriptions plus time to review dashboards
Evaluation upkeepTime to maintain test sets, plus model cost of running them
Prompt and knowledge maintenanceHours per month to update policies, examples, knowledge base
Model upgrades and migrationsPlanned migration testing, typically at least once a year
Human review and exceptionsShare of items needing review x minutes per item x loaded hourly cost
Support and improvementsBug fixes, small enhancements, vendor or developer retainer
Security and complianceAccess reviews, data retention, audits, legal review where needed
Total cost
Value deliveredHours saved, revenue recovered, errors avoided (be conservative)

For illustration only: an invoice-processing agent might have a moderate build cost, low monthly model usage because invoices are short and volume is in the hundreds, a noticeable human-review line in year one that shrinks as accuracy improves, and a planned migration cost each year. A customer-facing voice agent, by contrast, usually carries higher per-minute running costs and higher monitoring needs. The shape of the TCO tells you as much as the total.

Compare the total against the value line honestly. Our guide to measuring AI ROI without fooling yourself explains how to estimate value without inflating it.

Does buying an off-the-shelf AI tool avoid these costs?

Partly. A SaaS AI product bundles hosting, monitoring and model upgrades into its subscription, so those lines shrink or disappear from your worksheet. What doesn't disappear: per-seat or per-usage pricing that grows with you, the time to configure it and keep its knowledge current, human review of its output, and the cost of switching if it stops fitting. Custom systems carry more of the lines above but give you control over each one. Our post on whether to build or buy AI agents walks through that trade-off.

Frequently Asked Questions

How much does it cost to maintain an AI system per year?

It varies widely with usage volume, complexity and how much human review the work needs. Fill in the worksheet above with your own volumes rather than relying on a rule of thumb. For many SMB systems, running costs over a few years end up in the same range as the build, so plan for them from the start.

What is the biggest ongoing cost of an AI agent?

For high-volume or long-context systems, model usage usually leads. For lower-volume, high-stakes systems, human review and maintenance time often outweigh it. The worksheet makes it obvious which applies to you.

Do AI running costs go down over time?

Some do. Per-token model prices have tended to fall, systems get more accurate so less human review is needed, and optimizations reduce context and calls. Other costs rise with volume and data retention. Don't assume a net decrease; measure it.

How do I compare TCO between two AI vendors?

Put both through the same worksheet for the same volume and time horizon. Ask each vendor what's included, what's metered, what happens when models change, and what support costs after the first year. Pricing that looks cheap per seat can be expensive per task, and vice versa.

Should model upgrade costs be in the initial budget?

Yes. Model changes are a predictable event, not an emergency, so budget for at least one planned migration and regression test per year. Systems built with a clean model layer and a test set make this a small task rather than a project.

Want a realistic number before you commit?

If you're weighing an AI project and want the full cost picture, not just the build quote, walk us through the workflow and your volumes. We'll help you fill in the worksheet honestly and design the system so the running costs stay where you expect. See how we approach this in our AI automation work.