Wake up to a flood of news about OpenAI.
First, X user Leo@synthwavedd exclusively claimed: OpenAI has completed a new pre-training, codenamed "Bel," reportedly with a parameter scale exceeding 10 trillion, potentially the successor to Doug, and likely the foundation model for Astra / GPT-6...
A quick search reveals no confirmed news on that front. What is confirmed, however, is that OpenAI's first self-developed inference chip has delivered results.
Just now, OpenAI announced the latest test results for its first self-developed inference chip, Jalapeño: achieving a significant breakthrough. This chip, designed specifically for large language model inference, features a novel architecture that simultaneously increases throughput and reduces latency, achieving both while maintaining high energy efficiency. It has outperformed the efficiency of NVIDIA GB200 and GB300 systems in multiple model tests.
Shortly after, OpenAI CEO Sam Altman also stated on X: "We built a chip. It's fast."
It's important to note that as OpenAI's first self-developed inference chip, Jalapeño is not a chip for training the next-generation GPT model, but is optimized for the operational phase after the model is deployed.
Simply put: the training phase teaches the model its capabilities, while the inference phase allows the model to quickly answer user requests. Jalapeño primarily addresses the second issue.
OpenAI states that traditional hardware systems often face a trade-off between two metrics:
- Higher throughput: Process more requests per unit time.
- Lower latency: Provide users with faster responses.
For example, large-scale batch processing can improve overall efficiency but may increase the wait time for individual requests; while pursuing extremely low latency may sacrifice resource utilization. Jalapeño's goal is to optimize both within a single architecture.
Regarding specific performance, OpenAI states that in multiple large model inference tests, Jalapeño outperforms NVIDIA GB200 and GB300 systems. The tested models include: OpenAI GPT-OSS 120B; DeepSeek R1 670B; Moonshot AI Kimi K2.5 1T.
Particularly noteworthy is the appearance of DeepSeek R1 and Kimi K2.5, which seems to indicate that Jalapeño is not only optimized for OpenAI's own models but can also adapt to different large model workloads.
Interestingly, Jalapeño translates to "Mexican chili pepper," and this news immediately sparked jokes and discussions among netizens after its release.
"Are you planning to name your chips after chili peppers? Can't wait to try it."
For OpenAI, the launch of Jalapeño indicates that the company is attempting to integrate the entire chain of "model — software — chip — data center."
It's also worth mentioning that for this Jalapeño chip, OpenAI collaborated on the hardware with Cerebras.
Tibo posted: "This capability is thanks to our deep collaboration with Cerebras and their unique hardware architecture. In the future, this collaboration will further push the boundaries of 'ultra-fast' experiences.
I'm very much looking forward to continuing our work with Cerebras to keep pushing the limits on their platform, exploring how to run our most capable models at the fastest possible speed, and bringing that experience to customers with the highest performance demands."
In fact, OpenAI is not the first AI company to venture into self-developed chips. Many other major AI players have already entered the chip-making arena. For example, Google introduced TPU; Amazon developed Trainium and Inferentia; Microsoft is advancing Maia; Meta is also laying out its own AI accelerators...
The underlying logic is very similar: General-purpose GPUs are flexible enough but not optimized for all AI workloads. For companies running massive AI requests daily, if they can design chips tailored to their own models, software systems, and service methods, there is an opportunity to further reduce costs...
Let's take a closer look at Jalapeño's capabilities.
Faster, More Power-Efficient
Jalapeño's core advantage is completing more AI work under the same power consumption while returning responses faster. Existing systems typically trade off throughput and latency, whereas Jalapeño attempts to achieve both higher throughput and lower latency with the same architecture.
OpenAI emphasizes that this advantage is not only present with its own models but also extends to externally developed models, proving Jalapeño to be a more general-purpose inference architecture.
On the three public models GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T, Jalapeño achieved 1.5~1.9 times the AI work per watt at peak throughput, reducing end-to-end latency to 1/1.7~1/3.6 of the comparison systems; for highly interactive workloads, the performance advantage reached 2.1~4.1 times.
On GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T, Jalapeño's peak throughput per watt reached 1.9x, 1.7x, and 1.5x that of the current best systems, respectively.
Among them, on the largest model Kimi K2.5, Jalapeño's peak performance per watt increased by about 1.5x, and end-to-end latency decreased by about 3.4x.
On Kimi K2.5 1T, as the single-user decoding speed increases, Jalapeño's throughput per watt advantage compared to GB300 expands from the peak of 1.5x to a maximum of 56.1x.
OpenAI did not just compare the peak performance of a single chip but focused more on one metric: how much useful AI work can be completed per unit of electricity while meeting the latency requirements for users and interactive Agents.
The reason is also related to Agents. Agents often need to complete multiple steps consecutively; seemingly small delays in a single request can accumulate throughout the entire task execution.
This is also where Jalapeño's performance is most pronounced in low-latency ranges. Taking DeepSeek R1 as an example, under the conditions corresponding to the previous best TBT of the comparison system, Jalapeño's throughput per kilowatt reached 12,258 mixed TPS/kW, compared to GB300's 118, a difference of about 104.3x.
On DeepSeek R1 670B, Jalapeño's peak throughput per watt is about 1.7x that of GB300; at the same decoding speed, the advantage expands up to 104.3x.
OpenAI used the public benchmark InferenceX from SemiAnalysis for testing and compared it with leading commercial systems. From high-throughput services to highly interactive, low-latency scenarios, Jalapeño achieved better performance per watt and latency combinations on all three public models, sitting on the Pareto frontier.
At different operating points for DeepSeek R1 670B, Jalapeño forms a superior combination between throughput per watt and interaction speed/end-to-end latency, occupying the Pareto frontier.
Jalapeño's rated power consumption is 700W, though in the workloads tested this time, the actual sustained power consumption consistently remained at 550W or below.
Built for Agents
Jalapeño was designed from the ground up for current and future large language models, especially interactive Agents.
Different stages of LLM inference have different bottlenecks: Prefill relies more on computing power, Decode is more constrained by memory bandwidth, and data movement between cores and chips also increases latency.
Therefore, OpenAI co-designed the chip, memory, network, software, and rack-level system. Model states, including KV Cache, can be explicitly placed and kept local as much as possible, allowing Jalapeño to adapt to both Prefill and Decode and adjust as the workload between the two changes.
OpenAI believes this is precisely the key characteristic of Agent workloads.
Interestingly, AI is not just the entity Jalapeño serves; it also directly participated in the development of this chip.
Leveraging models at different stages, the OpenAI team took only 9 months from initial design to tape-out (finalizing the design for fabrication). Models were used to explore different implementations, shorten design, measurement, and verification cycles, and also participated in optimizing the chip's arithmetic circuits.
Jalapeño was also designed to be a clear, predictable programming target for both humans and models. Using the GPT-Astra-driven Codex, the team took only 2 months to achieve high-performance operation for three open-weight models that were not initially part of Jalapeño's production plan.
In some GPT-OSS Attention and MoE modules, the implementations generated by Codex were even 1.5~1.8 times faster than the original hand-written versions by human experts.
Deployment by Year-End, 2nd and 3rd Gen Already in the Pipeline
OpenAI plans to begin deploying Jalapeño into its own computing infrastructure by the end of 2026. Currently, the team is still conducting production validation, refining software, preparing for large-scale operation, and continuing to verify performance on more models.
Follow-up products are already underway, with Gen 2 entering deep development and Gen 3 beginning to take shape.
OpenAI summarized the efficiency changes brought by Jalapeño into three tiers: Ultra-fast mode can achieve the efficiency that only Fast mode had in the past; Fast mode can achieve the efficiency that only Batch mode had in the past; and Batch mode itself sees further efficiency improvements.
However, developing its own chips does not mean OpenAI is preparing to break away from NVIDIA.
OpenAI explicitly stated that meeting the ever-growing AI demand requires more computing power from all available sources. The company will continue to deploy NVIDIA and other partners' accelerators at scale, covering both training and inference workloads.
What do you think? Feel free to leave a comment and share your thoughts!
Reference Links:
https://x.com/OpenAI/status/2092300846675505602
https://x.com/sama/status/2092339694210040187
https://openai.com/index/jalapeno-first-results/
https://techcrunch.com/2026/08/25/openais-jalapeno-chip-is-built-for-fast-inference-at-scale-benchmarks-show/
This article is from the WeChat public account "Machine Heart" (ID: almosthuman2014), author: Machine Heart focused on AI, editors: Shan Hui, Youli
