What AI costs in production: budgeting beyond the pilot.

Why the pilot's bill says little about production's.

A pilot runs on a sample: a few hundred inputs, picked to show the idea works, run by the people who built it. Its bill is real, but it prices a different workflow from the one you are about to launch.

Four things change. Volume: production runs on every case, every day. Inputs: real cases carry longer documents, longer histories and inputs nobody planned for. Review: in a pilot the builders check outputs as part of their job, so nobody counts the time. Monitoring: a pilot has no alerting, no logs anyone reads and no test set re-run on each change.

Our guide to scaling an AI pilot to production covers the rest of that move. Cost gets one line there; this post covers it in full.

AI production costs, line by line.

Six lines cover a production AI workflow. Put each in the budget, even the ones that look small today.

• Model usage. The OpenAI, Anthropic and Azure OpenAI pricing pages list prices per million tokens, with input and output priced separately. On every text model we checked there in November 2026, output cost several times more per token than input. Data residency can add to the rate: OpenAI's pricing page shows a 10% uplift for regional processing on models released on or after March 5, 2026, and Anthropic's a 1.1x multiplier for US-only inference on Claude 4.6 and later models.
• Evaluation runs. Every prompt, model or retrieval change re-runs the test set before it ships: the test set's size times the cost per run, once per change.
• Logging and monitoring. Storing inputs, outputs and decisions for the audit trail, plus the alerts that tell someone a run failed.
• Human review. The minutes a person spends approving or correcting outputs, at that person's loaded cost. Nobody counted it in the pilot, so count it now.
• Engineering upkeep. Fixing prompts when inputs drift, handling vendor changes, keeping integrations working. A fixed number of hours a month.
• Model version changes. Vendors retire models. Each move to a new version means re-running the test set, reviewing the results and often prompt work. Monitoring, upkeep and model migrations after launch are what managed AI operations covers.

Estimating production cost from pilot data.

If the pilot logged token counts per run, its logs hold what you need. Pull three averages: input tokens per run, output tokens per run and review minutes per run. Then get one number from the business: runs per month at real volume. Use your own contract's price per million tokens, since list prices change every few months.

• Model cost per month = runs per month × (input tokens per run × input price + output tokens per run × output price) ÷ 1,000,000.
• Review cost per month = runs per month × share of runs reviewed × review minutes per run × loaded cost per reviewer minute.
• Fixed cost per month = monitoring and logging + evaluation runs for the changes you expect + upkeep hours × hourly cost.
• Illustration: 20,000 runs a month at 3,000 input and 500 output tokens per run comes to 60 million input and 10 million output tokens a month. Multiply each by your price per million. These are round numbers chosen for the example.

What pushes cost up at volume.

The estimate assumes the pilot's averages hold. Four things make them drift upward after launch.

• Longer context. The FinOps Foundation notes that the tokens a user types are not always the tokens billed, because applications add system prompts, history and retrieved content. Real cases carry more of all three. Some models also charge more for long prompts: OpenAI lists separate prices for inputs over 272K tokens, Anthropic charges more for prompts over 100,000 tokens on Claude Haiku 5.5, and Azure lists separate short and long context prices for some models.
• Retries. A malformed response that is retried costs a second run. Count retries per run.
• Agent loops. An agent that calls tools makes several model calls per task, and the count depends on the input. Each call sends the tool definitions again, and Anthropic's pricing page counts them as input tokens. Cap the steps per task and log the real count.
• New model versions. A replacement can carry a different price, write longer outputs or count tokens differently. Re-measure tokens per run after every change.

Ways to bring the cost down.

Each of these changes the price or the quantity, the two terms of the FinOps Foundation's basic equation: cost equals price times quantity.

• Trim the input. Send the fields the model needs instead of the whole record, and retrieve fewer, better passages. It is the cheapest change to try.
• Prompt caching. Anthropic's docs describe caching a repeated prompt prefix: writes cost 1.25 times the base input price for the default 5-minute cache, or 2 times for a 1-hour cache, and reads cost a fraction of it, set per model. Prompts under a per-model minimum length are processed without caching, with no error. OpenAI's and Azure's pricing pages list a lower price for cached input and, on some models, a higher price for cache writes.
• Batch processing. For work that can wait, Anthropic's Message Batches API charges 50 percent of standard prices, with results within 24 hours. OpenAI's Batch API and Azure OpenAI's (on Global Standard pricing) both give a 50% discount with a 24-hour window.
• Route simple cases to a smaller model. Send the easy inputs to a cheaper model and escalate the rest. The test set decides which cases count as easy.
• Set review by error cost. Review the outputs where a mistake is expensive, plus a random sample of the rest. Our post on human-in-the-loop AI workflows covers the routing and when review can drop.
• Provisioned capacity, once volume is steady. Azure OpenAI offers provisioned throughput with monthly and annual reservations. The FinOps Foundation warns that provisioned capacity is billed whether it is used or not.

Cost in the go or no-go.

Cost alone decides nothing. Put cost per run next to value per run: the minutes saved or errors avoided, measured against the baseline you recorded before the pilot. The FinOps Foundation also suggests cost per successful outcome: total cost divided by the runs that meet the quality bar.

For the go-live review, write down the cost per run you expect, the monthly spend that triggers a review, and the cost per run at which the workflow goes back to the manual path. Cost is one line of our AI production readiness checklist, signed by the budget holder.

If the numbers do not work, change the design before you drop the workflow: a narrower scope, a shorter input, a smaller model for the easy cases. Re-run the estimate after each change.

Do it yourself, or bring us in.

A team with token logs from its pilot and a recorded baseline can build this budget itself. The harder part is keeping it true after launch, as inputs grow, prices change and models are retired.

Outside help fits when nobody on the team has time for that work after launch. With managed AI operations, the same engineers who build your AI workflow keep it running in production: they watch token spend and error rates, swap models when a better or cheaper one ships, and fix prompts before a broken run reaches your team.

Frequently asked questions.

Keep reading.