Coinbase CEO Brian Armstrong shared an unexpected case: the company nearly halved its artificial intelligence costs despite exponential growth in token consumption. The key takeaway is that efficiency is achieved not through strict limitations, but by optimizing default settings, smart request routing, and aggressive caching.
Armstrong emphasizes that Coinbase engineers can choose any model, but default settings are crucial. The company is experimenting with using open-weight models as baselines, such as GLM 5.2 and Kimi 2.7, available through an internal gateway. Notably, 91% of employees never hit limits, so the company switched to cheaper settings rather than reducing quotas.
Routing, Cache, and Context Savings
Coinbase's internal architecture is built on pre-processing requests. The system automatically directs a task to the most suitable model, considering cache hit rate and cost. For example, an advanced model is used for planning, but it is excessive for routine tasks. Armstrong insists that the model selection should be automated by AI itself, not by a human.
The role of caching is highlighted separately. Cache misses are the easiest way to inflate costs. In the LibreChat service, the cache hit rate increased from 5% to 60% after proper configuration. Context savings are also important: Armstrong recommends starting new sessions when changing tasks, narrowly limiting file context, and disabling unused tools. The goal is not to spend fewer tokens, but to waste fewer of them.
Deutscher's "Barbell" Strategy
Analyst Miles Deutscher describes a similar approach, calling it "token engineering." His "barbell" strategy can reduce AI costs by 50% or more. He entrusts the first 10% of work and project planning to the smartest models (Opus, GPT), as this is a critical stage. The main 80% of routine tasks are performed by a cheaper open-source model. The final 10% and result verification are again assigned to high-level models. Deutscher has been using this scheme for several months and considers it the best way to reduce excessive AI spending.
Expert opinion: The Coinbase case is a brilliant example of a pragmatic approach to implementing AI in infrastructure. The market often chases the most expensive models, forgetting that 80% of tasks do not require such computing power. Automation of routing and aggressive caching are the "low-hanging fruit" that allow scaling AI usage without a multiple increase in budget. For crypto companies, where every cent counts, this is not just savings, but a matter of survival in a competitive environment.