Coinbase CEO Brian Armstrong revealed impressive results from AI cost optimization: the company nearly halved its AI expenses despite exponential growth in token consumption. This is a clear example of how sound engineering culture and a strategic approach can reverse the trend of uncontrolled cost increases.
Armstrong's key takeaway: savings are achieved not through rigid limits and bureaucratic control over spending, but through smarter default settings, intelligent request routing, and efficient caching. At Coinbase, engineers are free to choose any model, but default configurations play a decisive role. Moreover, 91% of employees never hit limits, so the company switched to cheaper settings without lowering thresholds.
Routing, Cache, and Context Savings
Coinbase's internal system preprocesses requests and directs them to the most suitable model, evaluating cache hits and cost. For example, an advanced model is needed for strategic planning but is excessive for routine execution. According to Armstrong, model selection should be automated by AI itself, not by humans.
Special attention is given to caching. Cache misses are the easiest way to inflate costs, so all requests are configured to reuse already processed information. In the LibreChat service, the cache hit rate increased from 5% to 60% after proper configuration. Context savings are also important: Armstrong advises starting new sessions when switching tasks, narrowly limiting file context, and disabling unused tools. The goal is not to spend fewer tokens, but to waste fewer of them.
Deutscher's "Barbell" Strategy
Analyst Miles Deutscher describes a similar approach, calling it "token engineering" and a "barbell" strategy. He entrusts the first 10% of work (project planning) to the smartest models (Opus, GPT). The core 80% of routine tasks are handled by cheaper open-source models. The final 10% and result verification are again assigned to high-level models. Deutscher has been using this scheme for several months and considers it the best way to reduce excessive AI spending.
My expert analysis: Coinbase's experience is not just a cost-saving case study but a paradigm shift. The market is moving from a race for the most powerful model to an era of intelligent orchestration. Companies that learn to effectively combine different models and adopt a culture of "token efficiency" will gain a massive competitive advantage. Those who continue to use LLMs as a "hammer for every nail" will face uncontrolled cost growth and loss of flexibility.