The one thing that changed the math
Most AI budgets were written around the wrong number. Teams priced the model — the license, the pilot, the training run — and treated “running it” as a rounding error. It isn't. The cost of running the model on every request is an operating cost, and it scales with the one thing you're actively trying to cause: adoption. The better the feature does, the more it costs. The demo was cheap because almost no one was using it.
Why it matters to you
This is the same pattern every infrastructure wave eventually teaches — you rent while you're learning, you own when the load is steady. What's emerging in 2026 is a hybrid split: keep experimentation and genuinely hard reasoning in the cloud where the newest chips live; move steady, predictable, high-volume work onto hardware you control, where a smaller model can run for a fraction of the rented frontier cost. Not anti-cloud. Not anti-frontier. Just matching each workload to the option that fits.
The three-question test
Before you send a workload to the biggest model by default, ask:
1. How hard is the reasoning? Routine work — classify, extract, summarize — runs fine on a small model. Save frontier models for genuinely open-ended reasoning.
2. How sensitive is the data? If you'd hesitate to email it to an outside vendor, that's a strong reason to keep the model on hardware you control — often the cleaner path to a compliance requirement, too.
3. How predictable is the volume? Spiky and experimental rewards renting. Steady and high-volume rewards owning. The per-request economics flip once utilization is high and constant.
One level deeper: the cost creep nobody models
Here's the part that isn't in the blog, and it's the one that bites teams six months in: your per-request cost isn't fixed. A feature that penciled out at 2,000 tokens a request rarely stays there. You add retrieval to make answers more grounded, you widen the context window, you paste in a few more examples to lift quality, you bolt on a guardrail check — and each of those quietly inflates the tokens per request. Volume didn't change; the unit cost did. I've watched features nearly double their per-request token count within two quarters without anyone deciding to spend more. So model the run cost — but treat it as a moving number, not a one-time calculation. Re-measure it quarterly, and put a tokens-per-request budget on the feature the way you'd cap a slow query. The teams that get surprised aren't the ones who never modeled it; they're the ones who modeled it once.
Do this one thing this week
Model the run cost of one AI feature at full volume — the whole organization using it, honest counts, your provider's real rate. Not the demo cost. That single number changes more roadmaps than any architecture diagram, and it's the difference between an AI feature that survives the budget review and one that quietly gets switched off.
On the blog this week: The Real AI Bill Isn't the Model — It's Running It. The long version of the three-question test, and why the cost that kills a project is always the one that scales with success. → dlegenddigital.com/blog
— Charles