If I were FinOps at Mistral AI

This is not a serious proposal for Mistral AI's stack. I take a problem its public record points to, make assumptions around it, and walk towards an answer with cost, latency and money in view. Everything runs on AWS, by choice. Mistral's real platform is its own, its announced sites are not on AWS, and I make no claim about them or about its bill.

The problem

Mistral has said more about its GPUs than most. A January 2026 post describes serving split into two kinds of instance: "P/D Disaggregated splits the processing of a query into two phases, processed by different instances". An inference job offer states the aim: "more throughput and lower latency out of every GPU under strict quality-of-service targets", and wants scaling that is "cheap and fast, not a performance cliff". The press reports 13,800 GB300 GPUs announced at one site, for training and inference, and Mistral Compute adds customers who "train and serve any AI workload" on the same kind of machine.

Mistral has never put it this way, so the question is mine: how do you divide GPUs between serving, training and tenants, when a GPU is the whole bill?

On an owned fleet, the answer is awkward: a freed GPU costs the same, and the saving exists only once the capacity is sold. On AWS, it is finally simple to state. A freed GPU saves money only if the instance is really terminated, or if something else uses its hours. Everything below is a version of that sentence.

What I assume

I do not price 13,800 GPUs. Nobody buys that many On Demand in one Region, and no list price means anything at that size. I price a slice of 200 instances, 1,600 GPUs, and keep the three-way split of training, serving and tenants at 50, 30 and 20. All of it is mine.

Low pairs the small fleet with the kindest lever assumptions, high the large fleet with the harshest, so the range is wide on purpose.

Where it runs decides the design, so I checked each GPU against the Paris offer file and its neighbours. Paris has one large training GPU: p6-b200.48xlarge, at $174.32 an hour. There is no P5, P5e or P5en in Paris, and the only other GPUs are G4dn at 16 GB and G5, G6 and Gr6 at 24 GB each. P5en exists in Stockholm and Spain. So serving stays in Paris, and anything that wants a P5en goes to Stockholm.

Version 0: the naive build

The first version is deliberately careless. Every tenant gets its own pool, because that is the easiest promise to make. Every pool is p6-b200.48xlarge, On Demand, in Paris, and nothing ever stops, because a GPU that has been hard to get is not one anybody lets go. Training reads its datasets from S3 Standard and writes checkpoints back, all through NAT gateways, since that is the default path out of a private subnet.

Scroll the diagram sideways →

GPUs are 98% of the bill. Each instance costs about $127,000 a month. So the first eight levers below decide the page, and the last four are housekeeping. That is the opposite of a normal cloud review.

Optimisation 1: batching, then really stopping

The problem. Serving runs 60 instances to carry the work of 24: 40% busy, sized for a peak that happens a few hours a day.

The change. Tune the batch limits, then scale with demand, and terminate what is not working. Continuous batching is how vLLM already works, so the gain here is tuning, not a new feature: I assume 20% more work per busy GPU, and a pool held at 55% busy on average. The trap is the word "freed". A stopped On-Demand p6-b200 may not come back when the peak does, and if you hold a reservation for the peak instead, you pay for it unused and the saving is gone.

Latency. More batching means more queueing at the peak. I assume about 100 ms more at p95, and the 55% is the level at which the latency targets still hold.

Dollars. About $3.01 million: 23.6 instances, all from really stopping. With no gain from tuning, the end of the page is $0.71 million higher. That leaves about $22.9 million.

Scroll the diagram sideways →

Optimisation 2: prefill and decode, sized apart

The problem. Prefill and decode want different things, and one replica count serves both badly under strict targets.

The change. Split them, as Mistral's post describes. I expected a throughput gain, and the sources do not give one. vLLM's guide says the split "won't necessarily move more tokens per second", and that what it buys is goodput: "the request rate you can sustain while requests still meet both their TTFT and ITL targets". So I price it as that: the pool holds 65% busy under the same targets, against 55% collocated. Both are assumed. It needs RDMA, which here means EFA in one zone.

Latency. This is the lever's real product, and it cuts both ways. "Every first token pays for the transfer", in vLLM's words. One test over EFA, on 8-GPU nodes, found inter-token latency up to 14 times better and time to first token worse, with collocated routing keeping the highest throughput. On latency-insensitive traffic, vLLM's advice is to stay collocated.

Dollars. About $713,000. If the targets are not binding, this lever pays nothing, and the end is $0.64 million higher. That leaves about $22.2 million.

Scroll the diagram sideways →

Optimisation 3: one shared pool, for tenants who accept it

The problem. Eight tenants hold five instances each, 30% busy. Each pool is sized for its own peak, and their peaks rarely meet.

The change. Move the tenants who accept logical isolation to the shared pool. I assume half of the work moves. The other half pays for isolation, and stays. That is a contract question first: a tenant that bought a machine bought exactly that machine.

Latency. A noisy neighbour is now possible. Per-tenant rate limits and a reserved share at the router hold it down, and I assume no change at p50 and some at p99.

Dollars. 20 instances become 7.7, which is about $1.57 million. That leaves about $20.7 million.

Scroll the diagram sideways →

Optimisation 4: a small-model tier behind a router

The problem. Every request runs on Mistral Large on a $174 instance, including the ones a much smaller model would answer.

The change. A router sends what it can to Mistral Small 4, in the same spirit as routing between the models of a family. I wanted this tier on cheap G6 nodes in Paris, and it does not fit. Small 4 is "119B parameters with 6.5B active". Its FP8 checkpoint holds about 121 GB of weights, and a g6.12xlarge has 96 GB across four GPUs. A g6.48xlarge, at $16.95 an hour, holds it as one replica across eight GPUs, with little left for a 256k context. NVIDIA's Transformer Engine lists FP8 on Ada, the L4's generation, so it would run, but it would be one replica stretched over a whole instance.

So the tier stays on p6-b200. The card's server example uses two GPUs, so an instance carries four replicas. Mistral's overview gives Large 4 52B active parameters against Small 4's 6.5B, a ratio of 8. That is a rough ceiling on the work per GPU, by my reading, and not a measurement, so I assume 5. I also assume 25% of Large's work moves, and 5% of that escalates back. The real test is quality per route, and it needs an evaluation.

Latency. A few milliseconds at the router, then a smaller model that starts faster (both assumed). An escalated request pays twice.

Dollars. About $735,000, after 1.5 instances of Small 4. If Small 4 does twice the work per GPU rather than five times, the page ends $0.39 million higher. At the ceiling of 8, it ends at $11.78 million. That leaves about $19.9 million.

Scroll the diagram sideways →

Optimisation 5: code on the small tier, not on Devstral Small 2

The problem. Code is a large share of what stays on Large, and much of it is mechanical (both assumed).

The change. I set out to run Devstral Small 2 here, and I do not. Mistral's model page says "This model is deprecated", and its overview lists it as deprecated on February 27, 2026 and retired on March 31, 2026. The alternative it names, Medium 3.5, is itself deprecated from October 31, 2026, and retires on April 30, 2027. The Apache 2.0 weights remain, so nothing stops anyone from serving them, but by the overview Mistral no longer does. Meanwhile Small 4, Apache 2.0 and generally available, unifies "instruct, reasoning, and coding" in one model. So the lever stands and the model changes: 15% of what is left on Large moves to the small tier, and 10% of it comes back.

Latency. Code turns on the small tier are faster per token, and each one that fails is a turn repeated (assumed).

Dollars. About $314,000. A separate Devstral lane, on top of the small tier, would add a second fleet and save nothing more, so that part is $0. That leaves about $19.6 million.

Scroll the diagram sideways →

Optimisation 6: ParallelCluster, FSx for Lustre and idle nodes

The problem. Training pays for 100 instances all month. Between runs, 15% of them do nothing, and during runs GPUs wait on S3.

The change. AWS ParallelCluster with dynamic nodes that scale to zero, and a scratch FSx for Lustre of 300 TB holding the working set. I assume read stalls fall from 6% of GPU time to 2%.

Latency. None for users. Jobs finish sooner, and a queue that has scaled to zero waits for capacity it may not get, which is what the next two levers are about.

Dollars. About $2.30 million, after $49,000 of FSx. FSx costs 0.4% of the end bill, and it pays for itself as soon as it saves half a percent of GPU time. That leaves about $17.3 million.

Scroll the diagram sideways →

Optimisation 7: Spot for interruptible training, in Stockholm

The problem. Evaluations, tuning and small fine-tunes run on the most expensive GPU in the Region, and they could survive an interruption.

The change. Spot queues, for jobs of eight instances, 30% of training. The Spot Instance Advisor has no row for p6-b200 in Paris or Stockholm, only in three US Regions, and Paris has no P5en, so these jobs go to p5en.48xlarge in Stockholm. The Advisor showed 80% off with interruption under 5% on the day I read it, a day's reading and no more. AWS says P6-B200 offers "up to 2x the performance compared to P5en", so I count two P5en for one p6-b200, the best case. A checkpoint of 6 minutes every hour costs 10%, and that overhead is in the figure. An interruption gives two minutes of notice, shorter than the checkpoint, so each one falls back to the last checkpoint.

The trap is the gang. At the top of that bucket, one instance is a small risk, and 64 together take 3.3 interruptions a month, with a 96.2% chance of at least one. So gangs stay off Spot. The fallback is a trap too: two P5en On Demand cost $135.45 an hour, more than the $113.67 of the next lever.

Latency. None for users. Jobs take longer per run, and data crosses from Paris at $0.02 per GB. I price 200 TB of that transfer, and no separate file system in Stockholm.

Dollars. $174.32 an hour becomes about $27.09 of Spot. It saves about $2.57 million. That leaves about $14.7 million. At a 60% discount and three times the interruptions, it still saves about $2.04 million.

Scroll the diagram sideways →

Optimisation 8: Capacity Blocks for training, and why a reservation pays nothing

The problem. Long training runs need capacity they are sure of, and the usual answers are not on offer. Paris has no Reserved row and no Savings Plan row for p6-b200. N. Virginia has both a lower On-Demand rate, $113.93, and a Savings Plan.

The change. Capacity Blocks at $113.67 an hour in Paris, 34.8% below On Demand and almost the N. Virginia rate, for training only.

Read the terms, because each one costs something. Duration runs from one day to 182, a block holds up to 64 instances, and an AWS Organization can hold 256 on a date. A block of 64 is not offered for every type in every Region, and I need about that in the base. The price is paid up front and "depends on available supply and demand", so the page's rate is a reference, not a quote. Cancellation is not allowed, and placement groups are not supported. An extension is "not guaranteed". No Savings Plan applies. And ParallelCluster manages a block's instances as static nodes, so inside a block the last lever's scale-to-zero is gone: idle hours are simply paid. I buy blocks around runs and assume 90% of the hours are used. The break-even against On Demand is 65.2%.

The bet I do not take. The 20 dedicated tenant instances run every hour, and the same rate would save about $886,000 a month at list on them, or $0.68 million off the end after the discount. I leave that out. AWS sells blocks for short work, "days or weeks at a time", and instances start terminating at 11:00 UTC on the last day. So an always-on tenant needs the next block in hand before this one ends, and the extension is not guaranteed. For training, a missed block is a delay. For a tenant, it is a production stop. Those 20 stay On Demand under an On-Demand Capacity Reservation, which guarantees the instances and saves nothing: it bills the On-Demand rate, used or not, and since they run every hour, it adds nothing either.

Latency. None, and the instances are placed in an UltraCluster.

Dollars. 57 instances' worth of work on 63.5 paid instances. It saves about $2.00 million. That leaves about $12.7 million.

Scroll the diagram sideways →

Optimisation 9: S3 tiers for checkpoints and datasets

The problem. Six petabytes of checkpoints nobody will reload, and eight of datasets, most of them untouched this quarter, all in S3 Standard.

The change. Expire what nobody needs: keep 15% of the checkpoints. Then move the cold datasets down: 60% go to Glacier Instant Retrieval at $0.005 per GB. Deep Archive is cheaper, and a 12-hour restore in front of a GPU that costs $127,000 a month is not a saving.

Latency. Glacier Instant reads in milliseconds. FSx in front hides the difference.

Dollars. About $194,000, or 0.75% of the start. Reading all the cold data back once costs about $144,000 at $0.03 per GB, so a dataset that returns every month does not belong there. That leaves about $12.5 million.

Scroll the diagram sideways →

Optimisation 10: NAT and data transfer

The problem. Three petabytes a month pass through NAT gateways at $0.05 per GB, on their way to S3 in the same Region.

The change. A gateway endpoint for S3, which has no charge. I assume 95% of the traffic moves.

Latency. One less hop, too small to notice against a checkpoint write.

Dollars. About $143,000, or 0.55% of the start. On most AWS bills this is a headline. Here it is an afternoon's work that rounds to nothing, and I still do it. That leaves about $12.4 million.

Scroll the diagram sideways →

Optimisation 11: Savings Plans, which pay nothing here

The problem. Steady compute is on On-Demand rates, and I want a commitment.

The change, and why I do not make it. There is nothing to buy. No Savings Plan rate for p6-b200 appears in the Paris offer, Spot does not qualify, and neither does a Capacity Block. A Compute Savings Plan would have covered a G6 tier, at 18.6% off, but that tier does not exist now that Small 4 sits on p6-b200.

Latency. None.

Dollars. $0. $12.4 million stands, and the $6.39 million of On-Demand p6-b200 has no commitment to buy.

Optimisation 12: negotiation

The problem. A company at this scale does not pay list, least of all here.

The change. A private discount, and I assume 8%. Nothing about any real agreement is public. I apply it to what is left except Spot and Capacity Blocks, because those are already market prices.

Latency. None.

Dollars. About $528,000, which leaves about $11.9 million. It is larger here than on most pages, because the largest line left is On Demand.

Where it lands

On-Demand p6-b200 is now $6.39 million, 54% of the bill, and Capacity Blocks are $5.27 million, 44%. The first six levers freed $8.64 million, a third of the start, and every dollar of that disappears if the instances keep running. Stopping is the whole game, and it is only safe if capacity comes back, which is why almost all of what remains is the price of keeping GPUs within reach.

One thing I left out: On Demand is 17.8% cheaper in Stockholm than in Paris, and a block costs the same. I keep serving in Paris for latency and for tenants' expectations, both assumed. The move would be a real saving for anyone who can take it.

The funnel

Your own numbers will differ, and yours can be measured. The calculator prices your stack line by line, and a FinOps audit starts from your real billing data, in the same order: reduce first, then commit, then negotiate.

What I cannot see from outside

• Anything about Mistral's real fleet. The 13,800 GPUs are an announcement, and I found no report of them in service. The slice here is mine.
• The busy shares. 40%, 30%, 55% and 65% rest on nothing public, and they decide the first three levers.
• Throughput. I measured nothing. Active parameters give a ceiling of 8 on Small 4's gain, and 5 is mine. If it does less, levers 4 and 5 shrink.
• How Large 4 fits. Mistral gives it 1.05T parameters. In FP8 that is most of a p6-b200's 1,432 GB, so one replica an instance is tight, and my serving counts assume it.
• Whether prefill and decode already carry production traffic. The post is from January 2026. It may be taken, and tuned.
• Whether 200 p6-b200 can be had in Paris at all. No page gives stock, I did not confirm that a block of 64 is offered in Paris, and a block's price is only known when it is offered.
• Whether a stopped instance comes back. My first lever assumes it does.
• What any tenant has signed. Pooling is a contract before it is a design.