Coinbase CEO Brian Armstrong shared impressive results from optimizing artificial intelligence costs. Contrary to popular belief, the company managed to nearly halve AI expenses while token consumption continued to grow. The key to success lies not in strict limits and total control, but in smart default settings, request routing, and efficient caching.
Armstrong noted that Coinbase engineers can choose any model, but default settings are crucial. The company is experimenting with using cheaper open-weight models such as GLM 5.2 and Kimi 2.7 through an internal gateway. Notably, 91% of employees never hit limits, so the company switched to cheaper settings instead of reducing limits. This preserved productivity while lowering costs.
Routing, Cache, and Context Savings
According to Armstrong, in Coinbase's own systems, requests are preprocessed and directed to the most suitable model based on cache hits and cost. For example, an advanced model is necessary for planning but excessive for execution. The AI itself should automate model selection, not a human.
Special attention is given to caching. Cache misses are the easiest way to drive up costs, so all requests are configured to reuse already processed information. In the LibreChat service, the hit rate increased from 5% to 60% after proper configuration.
Context savings were also highlighted as an important factor. Armstrong advises starting new sessions when switching tasks, narrowly limiting file context, and disabling unused tools. The goal is not to spend fewer tokens, but to waste fewer of them. This approach allowed Coinbase to nearly halve costs while consumption continued to grow.
Deutscher's "Barbell" Strategy
Analyst Miles Deutscher described a similar approach, calling it the era of "token engineering." He proposed a "barbell" strategy to reduce AI costs by 50% or more. He suggests entrusting the first 10% of work and project planning to the smartest models (Opus, GPT), as this is the most critical stage. The main 80% of routine work should be performed by a cheaper open-source model. The final 10% and result verification should again be handled by high-level models. Deutscher has been using this scheme for several months and considers it the best way to reduce excessive AI spending.
Expert opinion: Coinbase's strategy is a vivid example of how smart architecture and engineering culture can dramatically improve AI infrastructure efficiency. In an era where every token counts, the ability to properly route requests and use caching becomes a competitive advantage. I recommend that all companies implementing AI pay attention to these practices — they not only save budgets but also improve system speed.