FinOps for AI: tokens, cost per feature and the levers

FinOps for AI is FinOps applied to what AI costs: the same practice, engineering, finance and the business reading one set of cost data and deciding together what it is worth, on a bill counted in tokens rather than in instance hours. This guide sets out what the FinOps Foundation means by it, the units an AI bill is counted in, how to get from a token count to a cost per feature and per request, and the three levers that move an LLM bill most: routing, prompt caching and batch. A two-feature product priced before and after them shows the order and the size of each, and the last sections take the Framework's other capabilities in turn: allocation and showback, anomalies, forecasts and governance.

What FinOps for AI is

The FinOps Framework has treated AI as a technology category of its own since its 2026 revision, next to Public Cloud, SaaS and Data Center (Scopes and technology categories). Its page sets the problem in one sentence:

FinOps for AI focuses on addressing the cost complexity, faster development cycle, spend unpredictability

The Foundation keeps two subjects apart. FinOps for AI is the cost of AI: "This Paper covers FinOps for AI, managing the cost and value of AI" (FinOps for AI Overview). Using AI to do FinOps work is another topic, AI for FinOps, which this guide leaves out.

Three things change against cloud FinOps, in the Overview's words and the category page's:

• The unit. "Tokens, API calls, and provisioned throughput units behave differently from instance hours", and "One of the primary cost meters for AI usage is tokens."
• The volatility. Experiments start and stop quickly, so anomalies come more often and forecasts move more: "AI forecasts are less predictable than cloud forecasts, especially early on."
• Who spends. People who are not engineers now build AI systems too, so the practice has more people to reach and to teach than the engineering teams alone.

The principles, the phases and the capabilities stay the Framework's own (What is FinOps?). And almost every FinOps team now has this work. Asked which areas of spend they manage today or within 12 months, the State of FinOps 2026 respondents answered:

2024	31%
2025	63%
2026	98%

Source: State of FinOps 2026 Report

In short: FinOps for AI is the same practice on a bill counted in tokens, faster to move and harder to predict, with more people spending than the engineers alone.

Where AI spend shows up

The first task is to see all of it. The Overview lists six channels an organization buys AI through, each billed its own way, and asks for one AI scope over all of them:

At the centreOne AI scope

• Model provider APIs

  "Direct model provider APIs": billed per token, by the provider

• Cloud AI platforms

  "Cloud provider model services and AI platforms": Amazon Bedrock, Azure OpenAI, Vertex AI, on the cloud bill

• Routers and aggregators

  "Model routers and aggregators": one bill for several models

• SaaS and developer tools

  "Embedded AI in SaaS products and developer tools": seats or credits, often outside the cloud bill

• GPU providers

  "Cloud or specialized GPU providers for training and self-hosted models": "GPU hours, storage, networking"

• Your own infrastructure

  "Owned or colocated AI infrastructure": hardware, power and space

A product that calls a model through its cloud and pays for a coding assistant per seat has two of these on two different invoices. Only the first shows on the cloud bill, and even there a model can bill under its provider rather than under the cloud service: on Bedrock, Claude's charges appear "under the model provider (not under Amazon Bedrock)" (What Amazon Bedrock is).

The cost units of AI

An AI bill is counted in a handful of units, and a call bills several at once. The Foundation's token pricing paper starts from the first two: "Input tokens are the data you send to the model; output tokens are what the model generates", and output costs more because "Generating a response takes far more computation than reading a prompt". The units, and how the providers' own pages price them:

The rules come from Anthropic's pricing page ("The Batch API allows asynchronous processing of large volumes of requests with a 50 percent discount on both input and output tokens", and the multipliers "stack with other pricing modifiers, including the Batch API discount and data residency"), the Azure OpenAI pricing page (Batch "returns completions within 24 hours for a 50% discount on Global Standard Pricing"), Google Cloud's context caching overview ("Implicit caching provides a 90% discount on cached tokens compared to standard input tokens"), and the OpenAI API pricing page, which lists a cached input price, a Batch table and a Flex table for each model. Each page writes its own rates; this guide copies none of them into a comparison, because "Prices and ratios change often. Confirm current provider rates before using any figure in a forecast" (token pricing paper).

Two things the units hide:

• The tokens billed are not the tokens typed. "The tokens a user types are not always the tokens billed" (Overview): instructions, tool definitions, retrieved passages and the whole conversation so far are sent again on each call, and "Each turn adds to the input of every call that follows, so the cost per call rises as the conversation goes on" (token pricing paper).
• Input and output are not the whole bill. On Bedrock, "If you only sum input and output tokens, your totals will not match your bill": the Cost and Usage Report carries four token types, input, output, cache read and cache write, each a usage type of its own (Tokens: what one call counts).

In short: Read an AI bill by token type and by model, not as one total: output, cache writes and the prefix sent again on every call are where it grows.

Cost per feature and per request

A total says the AI bill rose; it does not say whether it rose for a good reason. The Framework's Unit Economics capability divides the cost by what it produced, and the AI category page and the Overview give the formulas for AI:

The Overview asks for these per use case: "Track cost per inference, per user, and per successful outcome for each use case". The last one is the one that decides: a cheaper model whose answers fail the quality bar more often can cost more per successful outcome, however low its price per token. It is also the hardest, since it needs the product to measure what a good answer is; cost per 1,000 requests per feature is where to start.

Measured per feature, the cost becomes something an architecture can be designed around. A decision taken at ISALYS Group, told on the cases page, reads:

"Managed frontier models for reasoning-heavy features, self-hosted lightweight Mistral models where data residency or per-token cost dominates. The trade-off: two runtimes and two security baselines, against lock-in and a token bill nobody could attribute. Cost per feature became a design input…"

A two-tier LLM runtime instead of one vendor, ISALYS Group

The example below makes that reasoning a bill.

A worked example: two features, one bill

An illustrative product, not a client case, with two LLM features on Amazon Bedrock, called from Paris through the EU geographic profile:

• An in-app assistant, reasoning-heavy: 400,000 requests a month on Claude Sonnet 4.6, each a shared prefix of 8,000 tokens (instructions, tool definitions, examples), 1,200 tokens of conversation and an answer of 350 tokens.
• A ticket tagger, high-volume and simple: 1.5 million support tickets a month, 900 input tokens each and a label of 30 tokens, read by a report the next morning, so it can wait.

Both start on the same model. The three levers this site names for AI bills, on the home page's AI use case, routing, prompt caching and batch, are then applied in that order, each on top of the one before:

From $19,651.50 to $7,030.65 a month, 64.2% less, computed. The two months line by line:

A · One model for both features

One model for both features Illustrative

Line	A month
Compute	$19,651.50
Assistant input: 400K × 9,200 tokens3,680 M tokens × $3.30 /M tok	$12,144.00
Assistant output: 400K × 350 tokens140 M tokens × $16.50 /M tok	$2,310.00
Tagger input, Sonnet 4.6: 1.5M × 900 tokens1,350 M tokens × $3.30 /M tok	$4,455.00
Tagger output, Sonnet 4.6: 1.5M × 30 tokens45 M tokens × $16.50 /M tok	$742.50
A month, computed	$19,651.50

B · After the three levers

Routing, prompt caching and batch Illustrative

Line	A month
Compute	$7,030.65
Prefix written to the cache: 40K × 8,000 tokens320 M tokens × $4.125 /M tok	$1,320.00
Prefix read from the cache: 360K × 8,000 tokens2,880 M tokens × $0.33 /M tok	$950.40
Assistant turn, input: 400K × 1,200 tokens480 M tokens × $3.30 /M tok	$1,584.00
Assistant output: 400K × 350 tokens140 M tokens × $16.50 /M tok	$2,310.00
Tagger input, Haiku 4.5, batch: 1.5M × 900 tokens1,350 M tokens × $0.55 /M tok	$742.50
Tagger output, Haiku 4.5, batch: 1.5M × 30 tokens45 M tokens × $2.75 /M tok	$123.75
A month, computed	$7,030.65

After the three levers costs $12,620.85 a month less than one model for both features (64.2% less), computed.

USD before tax, 730 h a month. AWS Price List, eu-west-3, offer file published October 12, 2026, fetched October 16, 2026. Illustrative configuration, not a client case.

The same months by feature, the showback a product team would read:

Before the levers, the tagger is 26.4% of the bill on work a smaller model may do; after them, the assistant is 87.7% of it, computed, and the next decision is its own. What each lever does, and what it asks for:

Routing: the smallest model the evaluation accepts

Sending the tagger to Claude Haiku 4.5 is the routing step. It rests on one assumption the price list cannot check: that the smaller model tags as well. Only an evaluation on the product's own tickets says so, and "Cost of results that meet the quality bar" is the measure. Routing can be done by the application, one model per feature as here, or per request by a router; on Bedrock, intelligent prompt routing routes between the models of one family, at a price per 1,000 requests on top of the tokens.

Prompt caching: the prefix that repeats

The assistant sends the same 8,000 tokens on every request. Cached with a 5-minute time to live, the prefix is written by the 10% of requests that find the cache cold and read by the others, and the caching step is the largest of the three here. The Foundation names the condition: "Use prompt caching for stable prompt prefixes" (Overview), and the risk: "If cached content is not read back enough times before it expires, the write premium costs more than caching saves" (token pricing paper). The Bedrock article prices a cache by its share of cold requests (Cache the prefix that repeats). The KPI is the cache hit rate, the "Share of input tokens served from cache".

Batch: what can wait

The tagger's labels are read the next morning, so its requests can go to batch: the same model at the batch rates. The Overview's rule: "batch processing for work that is not time sensitive". Batch takes things away for its price: on Bedrock, prompt caching "is not supported with the batch inference API" (prompt caching), which is why the assistant keeps its cache and the tagger, which has no shared prefix worth caching, goes to batch. The Bedrock article compares batch with the Flex tier (Pick the tier by how long the answer can wait).

In short: One model per feature, chosen by evaluation, a cache on the prefix that repeats and batch for what can wait: in this example the cache saves most, the routing next and batch the least.

Allocation and showback

Cost per feature needs to know which feature made each call, and AI makes that harder: "More acute are the challenges of identifying the consumer of the model output" (FinOps for AI). The Overview's first rule is simple: "For API-based services, map each API key to one team, application, and cost center". On Bedrock, the equivalent is an application inference profile, which "carries cost allocation tags that flow to AWS Cost Explorer and AWS Cost and Usage Reports" (See the bill by model, token type and application); the tags then work as the Cost Explorer article describes.

Then show each team its line before billing it: "Report through showback first", since "Visibility alone often changes behavior" (Overview). Chargeback comes later, for shared models and platforms. Two cases remain open: a shared model serving several products needs a split rule, and an agent that calls other agents has no settled one yet, the category page noting a "Lack of generally accepted frameworks for cost allocation across multi-agent workloads."

Anomalies

"AI anomalies carry more risk and happen more often than in typical cloud spend" (Overview): a prompt that grows with each turn, a loop of agent steps, a batch job sent to the real-time tier. The category page adds that the criteria are harder to set, with "Higher volatility and more difficulty in establishing correct criteria for defining anomalies", and gives a measure for the result:

Daily spend alerts are too slow for a token bill. The Overview's advice: "Monitor tokens per minute at the application level, not only daily spend." On Bedrock, the token counts of every call are CloudWatch metrics, per model, cache reads and writes included (Tokens: what one call counts), and AWS's Cost Anomaly Detection watches the bill behind them.

Forecasts

An AI forecast is built from its drivers, not from last month's trend. The Foundation's forecasting paper applies "the quantity * rate = cost forecasting approach" to each kind of AI cost (How to Forecast AI Services Costs in Cloud): requests a month, times tokens per request by type, times the rate of the model and the tier each feature uses. The worked example is that calculation, and every lever changes one of its terms.

• Revise often. "The weekly/monthly rolling forecast cadence in AI is ideal", in the same paper, since the category page warns that "Higher volatility of costs makes trend-based forecasting in general more challenging".
• Forecast the model change. A new model can change the rate of a feature overnight, up or down, and providers retire models on their own calendars: a forecast names the model behind each line.
• Keep optimization in the forecast. A cache or a routing decision taken next month lowers the run rate, which a trend line will not see.

Governance

"Govern without blocking" is the Overview's rule for Governance, Policy & Risk: "set approved model lists, quotas, rate limits, and budget thresholds", and "Allow lighter rules for pilots so governance does not push teams to work around it." The controls tighten as a use case moves from pilot to production, which the Overview draws as Crawl, Walk and Run:

1. Crawl

   "Fail fast. Set cost and time limits up front." Pilots, with a cost ceiling and an end date.

   1

2. Walk

   "Fund what production use requires and no more." Cost per feature known, showback in place.

   2

3. Run

   "Keep costs proportionate to the value delivered." Cost per successful outcome tracked per use case.

   3

The same staging applies to capacity. Provisioned throughput, the Reserved tier on Bedrock, provisioned throughput units (PTUs) on Azure OpenAI, Provisioned Throughput on Vertex AI, is a commitment: on Azure "You are charged an hourly rate per model regardless of usage" (Azure OpenAI pricing), and on Vertex AI it is "a fixed-cost, fixed-term subscription available in several term-lengths" (Provisioned Throughput overview). The Foundation's capacity paper is plain about the risk: "You are paying for this capacity 24/7, whether you use it or not", and "If your traffic is highly variable… provisioned capacity may be a trap." Reserve last, from measured tokens per minute, after the levers above have shrunk them: in Paris, a 3-month Bedrock reservation costs less than on-demand tokens only above 90.0% average use, computed (Commit only to measured, steady capacity).

In short: Allocate every call, show each team its line, watch tokens per minute, forecast from drivers, and keep commitments and strict rules for the use cases that have earned them.

Where to start

For a team with an AI bill and no practice on it yet, five steps, in order:

1. 1

   Find every place AI is billedInform · Data Ingestion

   Model APIs, the cloud's AI services, SaaS seats, GPUs: list them in one AI scope, with their token types, before reading any total.

2. 2

   Give every call an ownerInform · Allocation

   One API key or one application inference profile per feature and team, then showback: each team sees its own AI line.

3. 3

   Divide by the unitQuantify · Unit Economics

   Cost per 1,000 requests per feature first; cost per successful outcome once the product measures what a good answer is.

4. 4

   Pull the three levers, in orderOptimize · Usage

   Route each feature to the smallest model its evaluation accepts, cache the prefix that repeats, send what can wait to batch.

5. 5

   Commit last, and govern lightlyOperate · Governance

   Reserve capacity only for measured, steady traffic; set approved models, quotas and budget alerts, with lighter rules for pilots.

To learn it formally, the FinOps Foundation has a certification on it, FinOps Certified: AI Value; its price and its place among the other certifications are in FinOps certification.

FAQ

What is FinOps for AI?

FinOps applied to the cost and value of AI: the FinOps Foundation's practice and Framework, used on a bill counted in tokens, API calls, provisioned throughput and GPU hours. In the Foundation's words, it addresses "the cost complexity, faster development cycle, spend unpredictability" of AI (What FinOps for AI is).

How is an LLM bill calculated?

Per token, by type: input, output, cache writes and cache reads, each at the rate of the model and of the tier or mode used (standard, batch, provisioned). Output costs more than input, and the instructions and conversation sent again on each call are input too (The cost units of AI).

What is the best metric for AI cost?

The Foundation's own: cost per successful outcome, the "Cost of results that meet the quality bar", per use case. Cost per 1,000 requests per feature is where to start, since it needs only the bill and the request count (Cost per feature and per request).

What saves the most on an LLM bill?

It depends on the workload. In this guide's illustrative two-feature product, prompt caching of a long shared prefix saved most, routing a simple feature to a smaller model next, and batch the least, 64.2% together, computed (A worked example).

When is provisioned throughput worth it?

For steady traffic that keeps it busy, or for latency the shared tier cannot give. It is billed every hour whether used or not, and for highly variable traffic the Foundation calls it "a trap" (Governance).

Is there a FinOps certification for AI?

Yes: the FinOps Foundation's FinOps Certified: AI Value (Where to start).