Coinbase CEO pushes AI cost cuts, shifts to Chinese LLMs, token usage

Coinbase announced that AI spending has dropped to roughly 50 % of its peak even as token consumption climbs to one of the highest levels in company history, a shift driven by new routing rules and a move to cheaper Chinese large‑language models (LLMs) [1][2].

New AI‑cost strategy

CEO Brian Armstrong outlined five tactics to keep AI costs low while encouraging engineers to “token‑max” responsibly. The first tactic replaces default frontier models with open‑weight Chinese alternatives—GLM 5.2 from Z.ai and Kimi 2.7 from Moonshot AI—whose per‑token fees are substantially lower than those of Anthropic or OpenAI [1]. A second tactic routes prompts automatically to the most appropriate model based on task difficulty, reserving expensive frontier models for planning‑heavy work and using cheaper models for execution [1].

Impact on spend and usage

Armstrong shared a graph (timeline unspecified) showing token usage at a near‑record level while AI spend has fallen to “nearly half its peak” [1]. The shift to Chinese LLMs, combined with an automatic routing system, boosted the cache‑hit rate from about 5 % to 60 % and cut overall AI costs in half, according to the company’s internal data [2]. The higher cache efficiency means fewer inference calls, directly reducing per‑token expense.

Organizational context

The AI‑cost reforms arrive less than two months after Coinbase trimmed roughly 14 % of its workforce—about 700 roles—in an AI‑driven restructuring [3]. Armstrong framed the layoffs as a move toward a “leaner, faster, and more efficient” operation, emphasizing a flatter hierarchy and “AI‑native” talent [3]. The broader industry trend is moving away from the “token‑maxxing” approach that allowed unlimited token consumption, toward accountability mechanisms that tie AI spend to measurable business output [1][2].

What to watch

• Cache‑hit rate: A sustained 60 % hit rate would suggest the routing and caching changes are effective; a dip could signal performance or cost pressures.
• AI spend trend: Monitoring quarterly AI‑budget reports for further reductions or rebounds will indicate whether the cost‑control measures stick.
• Token usage levels: Continued growth toward historic highs could pressure the company to refine usage caps or introduce new efficiency tools.

Coinbase’s pivot to cheaper, non‑Western LLMs illustrates how crypto‑focused firms are re‑engineering AI consumption to preserve margins, raising questions about the long‑term viability of domestic AI providers in high‑volume enterprise settings.