For two years the model question had one answer: use the best available. That was the correct answer when the gap between the frontier and everything else was wide, and when most of what people asked models to do was open-ended enough to need general capability. Both of those conditions are weakening, and open-weight models are the reason the default deserves another look.
Azeem Azhar collects the evidence in a recent issue. Bridgewater, working with Thinking Machines, fine-tuned an open Qwen model on expert-labelled data for its internal information-filtering task and beat every frontier model it tested: roughly thirty per cent fewer errors than the best closed model, at one-fourteenth of the inference cost. Vercel’s open-weight token share hit sixty-two per cent in a single day, up from twenty-eight per cent two months earlier. Thomson Reuters has built its first in-house model on Qwen specifically to cut costs.
Why a small tuned model wins on a narrow task
A repeated, well-defined task does not need general intelligence. It needs to be reliably right on that one task. General capability is what you pay the frontier premium for, and on a narrow job most of that capability sits idle.
Fine-tuning a small model on your own labelled examples buys the thing you actually need: consistent performance on the specific distribution of inputs you see, rather than broad competence you are not using. And the models in question are small enough to matter operationally. A 27-billion-parameter model runs on a workstation. The inference bill for a high-volume production path drops by an order of magnitude, and the data stays on hardware you control.
The portfolio question changes
If you run a model portfolio, the question to ask about each workload shifts. Not what is the best model, but what is the cheapest model that clears the bar on this specific task.
For the high-volume, well-scoped production paths, the answer is increasingly a tuned small model. Classification, extraction, routing, filtering, structured summarisation: these are jobs with a definable right answer and enough volume to justify the tuning effort. Running them on a frontier model is paying for a capability the task never exercises.
A concrete split: suggesting the right macro for an incoming support message is a tuning candidate, because you have years of labelled examples and thousands of cases a day. Drafting the reply to an unhappy enterprise customer who is threatening to leave is not, because the inputs are open-ended, the stakes are high, and you run it rarely. Same product, same team, two different model decisions, and the difference is legible once you ask what the task actually demands.
What still earns the frontier premium
Three scenarios, each real enough to override the default.
Genuinely open-ended work, where you cannot enumerate the task in advance and you need the model to handle inputs you did not anticipate. Tuning a small model for a job you cannot specify does not work.
Low-volume, high-stakes decisions where cost is irrelevant and you want the best available judgement on every call. If you run something a hundred times a month and the downside of a bad answer is large, use the best model and stop optimising.
The operational envelope. Azhar’s point is that frontier labs still win substantial business on reliability, latency guarantees, harness quality, and support, not on raw model performance alone. A cheaper model that you now have to host, monitor, and keep current is not cheaper if you lack the team to do that well.
The prerequisite most teams skip
Tuning a small model needs labelled data and an evaluation that defines correct on the task. So does knowing whether the tuned model is good enough to trust in production.
If you cannot articulate what a right answer looks like for a workload, you cannot tune for it and you cannot measure it, which means you are stuck defaulting to the frontier because it is the safe-looking choice. The discipline of writing down the task and its success criteria is what unlocks the cheaper option. Teams that have done that work have a real decision to make. Teams that have not will keep paying the premium by default.
This is also where the two halves of the cost problem meet. The labelled data and the evaluation set that let you move a workload to a small model are the same artefacts that tell you whether any model is doing the job well. The work you do to cut the bill is the work you should have done anyway to know the system is trustworthy.
A cost story you can opt out of
“AI is essentially free” was never true, and it is less true now that agentic workloads have pushed inference volumes up. But “AI must mean the frontier model” is also a cost story, and for most of what a company runs in production, it is one you can opt out of.
For most production workloads, the right model is now the smallest one that clears the bar. Reach for the frontier when the task genuinely earns it.
Leave a Reply