OpenAI and Broadcom unveil Jalapeño as AI inference chip race heats up

OpenAI has shown its first custom AI processor built with Broadcom. The chip is already in testing and is expected to reach customer workloads later this year, as investors look beyond GPU sales to the cost of running AI models.

OpenAI disclosed its first custom AI processor, called Jalapeño, on June 24. Axios, The Verge and MarketWatch reported that the chip was developed with Broadcom and is aimed at inference, the compute work that happens after a user asks a model to answer, write code or generate media.

Until now, OpenAI has mainly expanded compute by buying, renting and testing outside hardware. Nvidia GPUs remain central, while Google TPUs and Cerebras systems have also been evaluated. Jalapeño moves OpenAI into another part of the stack: designing silicon around its own model workloads.

Nine months to a sample

The Verge reported that roughly nine months passed between the Broadcom partnership being announced and the Jalapeño reveal. OpenAI has begun testing samples and expects to put the chip into customer service later this year.

Broadcom Chief Executive Hock Tan described the processor as the first step in a multi-generation computing platform. Public reporting has referred to performance-per-watt and cost advantages, but details such as process node, power draw, foundry partner, price and full benchmarks have not been released.

Claims that the chip could cut costs by about 50% compared with traditional options remain unverified in public data. The problem OpenAI is trying to solve is clear enough: after a model has been trained, every search, code completion, image request and voice interaction still carries a compute bill.

Nvidia remains hard to displace

Nvidia still controls much of the infrastructure used for large-model training, from general-purpose GPUs and the CUDA software ecosystem to experience running massive clusters. Jalapeño does not mean OpenAI is about to leave Nvidia behind.

Requests to ChatGPT, Codex and other AI services create a high-volume stream of relatively repeatable compute. Nvidia and AMD can still handle training and flexible workloads, but a successful OpenAI ASIC could divert some cloud inference growth toward customer-owned chips.

Broadcom moves deeper into AI infrastructure

Broadcom has already worked on custom ASIC projects, including Google TPU-related work. Jalapeño pushes the company further into OpenAI's supply chain and broadens its AI revenue exposure beyond networking chips and switches.

MarketWatch also reported that Celestica is involved in related hardware infrastructure. Broadcom's role is not to run an AI cloud, but to turn a large customer's design needs into silicon and systems that can be manufactured and deployed.

AI hardware stocks split

AI hardware stocks did not move in one direction after the announcement. Investors Business Daily reported that Broadcom rose 0.5% on June 24 to $382.07, while Nvidia slipped 0.5% to $199. Newly listed Cerebras fell 19.6% to $182.26 after its first earnings report.

Cerebras reported 94% year-on-year revenue growth for the first quarter and raised its full-year revenue outlook. Investors focused instead on falling gross margin. Its customer list includes OpenAI and AWS, keeping customer concentration in view.

On Wall Street, the AI hardware trade is no longer only about who sells more GPUs. Investors are also watching inference costs, margins and customer concentration. Nvidia remains the main supplier for training and general AI compute; Broadcom benefits from customer-designed chips; Cerebras and other challengers have to show that fast growth can become steadier profitability.

The cost test

Usage of ChatGPT, Codex and other AI services continues to rise, and OpenAI has to pay for the compute behind every request. If a custom inference chip can be deployed at scale, OpenAI would gain more room on pricing, product cadence and supply chain planning.

Jalapeño also puts OpenAI into a longer hardware cycle. If model architectures change quickly, the chip will have to balance efficiency with flexibility. For now, it is best read as a supply-chain move to reduce inference cost, not as an immediate change in Nvidia's position.

Sources: This article draws on reporting from Axios, The Verge and MarketWatch on OpenAI and Broadcom's Jalapeño processor, and Investors Business Daily on Broadcom, Nvidia and Cerebras share moves. Some market chatter has suggested a large cost advantage for Jalapeño, but full public benchmark data has not been released; this article does not treat any specific cost-reduction figure as confirmed. References: Axios, The Verge, MarketWatch, Investors Business Daily.