OpenAI Jalapeño Chip Outperforms Nvidia GB300 in AI Inference

OpenAI claims its custom-designed Jalapeño AI chip has outperformed Nvidia’s GB300 processors in key tests, delivering 1.5 to 1.9 times more AI work per watt and reducing end-to-end latency by 1.7 to 3.6 times across several AI models [2]. This move aims to reduce the company's reliance on Nvidia and lower the cost of running its AI services [1, 4].

Performance and Purpose

The Jalapeño chip, developed in collaboration with Broadcom and manufactured by TSMC, is an application-specific integrated circuit (ASIC) designed exclusively for AI inference [2, 3]. Inference is the stage where a trained AI model processes user requests and generates responses, a critical and costly component of AI services like ChatGPT [3, 4]. OpenAI's chip chief, Richard Ho, stated that Jalapeño performed better than the Nvidia GB300 in tests measuring AI workload per power unit and response speed [1].

The chip's sustained power consumption is at or below 550W, though rated for 700W [2]. OpenAI claims Jalapeño can deliver strong performance while using around 700 watts of power, which could reduce the operational costs of large data centers [1]. Broadcom CEO Hock Tan further claimed Jalapeño matches the performance of Nvidia’s Blackwell architecture and Google’s TPU, offering approximately a 50% cost advantage on a per-token and per-kilowatt basis [2]. The chip was designed with OpenAI's own AI models assisting in the process, going from schematic to tape-out in about nine months [2, 3].

Strategic Implications

OpenAI plans to begin using its Jalapeño chip to run its AI models later this year [1]. A limited deployment is expected by the end of 2026, with a full-scale rollout anticipated in 2027 [2]. The company is already developing a second-generation Jalapeño chip, with design expected to be completed in the coming months, and has started planning a third generation [1, 2]. This strategy aims to lower the cost of building and operating its expanding AI infrastructure [1].

This move aligns with a broader trend among major AI players, including Google with its Tensor Processing Units (TPUs) and Amazon with its Trainium and Inferentia chips, to reduce reliance on Nvidia's general-purpose GPUs for cost-sensitive workloads [2]. While Jalapeño is designed for inference, Nvidia's processors are expected to remain strong in AI model training [1, 2]. OpenAI has stated it will continue to purchase hardware from Nvidia, AMD, and other suppliers to meet its computing needs [1, 2].

What to watch

• Deployment timeline: Monitor the limited deployment of Jalapeño by the end of 2026 and the full-scale rollout expected in 2027 [2].
• Next-generation chips: Watch for updates on the design completion of the second-generation Jalapeño chip in the coming months and plans for a third generation [1].
• Competitive response: Observe how Nvidia and other chip manufacturers respond to OpenAI's performance claims as Jalapeño enters broader use.

The success of Jalapeño in real-world data centers could significantly impact the economics of AI services by reducing the cost of inference, a key factor in making AI products profitable at scale [3].