Why did the headline metric shift from accuracy to survival?
In one frontier model's case studies, the metric being celebrated has subtly changed. It is not benchmark scores but the ability to survive in messy environments. The most striking is Cognition's testimony. One bank customer using Devin—the world's first AI software engineer—has 30,000 engineers and runs internal tools accumulated over decades. Until now, models completely broke down in such environments.
The new model's leap came precisely from the ability to hold up in that environment. Cognition calls it 'surviving in these environments'—not collapsing inside a real codebase piled with layers of legacy, and thereby earning enough trust that the customer will hand over bigger projects. That is the direction in which the frontier has moved.
How is 'answering' different from 'holding up'?
The other cases share the same grain. Cursor says it used to have to keep nudging the model, but now, told only to 'go do it,' the model grabs a deep problem and keeps going on its own. Holding course without intervention—this is the decisive difference between a short answer and long work.
Base44 is more concrete. A system-prompt rewrite that would have taken their top three engineers a couple of days, the model ran for four hours and returned 90–95% of what was needed. The four hours is itself the metric, because it measures the ability to push a long autonomous run through to the end, not to produce a single response.
Legal (Thomson Reuters) and finance (Hebbia) show another axis. Legal motions a lawyer spends days or weeks on; financial data that must always be right. These, they say, required far too much precision and context to be possible with earlier models. Holding up in domains that must 'always be right' is a different kind of trust from accuracy.
What was the real reason enterprises couldn't use AI?
These cases point to a shared conclusion. The real reason enterprises couldn't put AI to serious use was not that the model wasn't smart enough, but that it couldn't withstand the legacy and exceptions of reality. It already solved clean benchmark problems well, yet broke down in real environments tangled with thirty-year-old internal tools and countless exception cases.
So the definition of the frontier moves—from 'can it answer the question' to 'can it hold up for a long time amid the chaos of real work.' Once this wall begins to fall, the premise of every adoption discussion that was deferred with 'the model isn't there yet' changes.
So where does the AX bottleneck move?
From an AX (AI transformation) standpoint this is a big signal. The bottleneck now moves from the model's intelligence to how we hand it our organization's messy context. If the model can survive in the environment, the remaining task is to organize that environment into a form the model can read and pass it over.
Concretely, three things. First, organizing our legacy and exceptions so the model can access them. Second, designing the boundaries and verification points that make a long autonomous run safe to delegate. Third, drawing the line—in work that must 'always be right'—for how far to trust the model and where a human checks. The stage of waiting for the model to get smarter is passing. Now the speed of organizing our own context decides the speed of adoption.