[Introduction] GPT-5.6's reasoning speed is shockingly high at 750 tokens/second! A professional insider reveals: It will run across 100 wafers. AI is changing from thinking to flashing, is the era of real-time intelligence really here?
According to various leaks, GPT-5.6 is about to be open to the public.
Recently, various speculations about this model have been trending on X.
On June 26, OpenAI officially announced the new generation GPT-5.6 family.
Moreover, there was this sentence in the official blog: OpenAI plans to launch a new cutting-edge model — GPT-5.6 Sol — on chip giant Cerebras's custom hardware this month, with a reasoning speed reaching a terrifying 750 tokens per second!
This means that complex Agent operations that used to require minutes of waiting can now be completed in the blink of an eye.
Clearly, OpenAI has taken the first disruptive step in hardware and model co-design.
Coupled with the recent exposure of the first self-developed AI inference chip Jalapeño, we can sense that OpenAI already has the ambition to become a full-stack AI empire.
Speed is Supreme in All Skills: The Dimensional Strike of 750 Tokens/s
What does "750 Tokens per second" mean?
For humans, this is equivalent to reading and outputting about 500 to 600 Chinese characters per second.
The text you are reading now, GPT-5.6 Sol can generate in less than a few tenths of a second.
On X, renowned developer Caleb Shepherd excitedly stated: "This is what I'm most excited about, GPT-5.6 Sol running on Cerebras. It's not just that coding becomes faster, but the speed of computer usage undergoes a qualitative change. We no longer have to wait two minutes for AI to click a button."
For a long time, although large models have become smarter, "reasoning latency" has been the biggest bottleneck for deploying real-time interactive multi-step Agent tasks.
When models grow to have trillions of parameters, traditional GPU clusters often encounter physical bottlenecks in inter-node communication (NVLink interconnects).
OpenAI's answer is: Don't make the model adapt to the hardware; make the hardware and the model integrate into one.
According to preliminary information disclosed officially, GPT-5.6 Sol will be opened to specific customers in an extremely limited scale in July, gradually expanding as production capacity ramps up.
As many guessed online, this is definitely an extremely expensive service, a privilege tailored for top-tier enterprises willing to pay for speed.
How to Fit a 3 Trillion Parameter Beast into a Chip?
When the news of 750 Tokens/s came out, LLM Arena's lead Peter Gostev raised a question everyone was puzzled about:
What exactly is going on with GPT-5.6 Sol on Cerebras? As far as I know, this seems to be the complete same model (including visual and other multimodal capabilities), not a stripped-down version like the previous GPT-5.3-Codex-Spark which lacked vision and context.
But my understanding is that Cerebras's single chip can only hold a model with at most 700 to 900 billion parameters. So, has the model shrunk? Or is there a new type of chip I don't know about? Or is it some new multi-chip collaboration technology?
This doubt immediately sparked discussions among many netizens.
Some joked that everyone was doing a "forensic-level chip audit at midnight," saying, "If this is really the same complete model, it's like someone forced a super yacht into a glass bottle and didn't tell you how they did it."
Soon, senior technical expert Bleys Goodson provided a highly convincing hardcore deduction —
GPT-5.6 Sol is not stuffed into a single chip, but spans 70 to 100 Cerebras wafer-scale chips!
The Ultimate Deployment Aesthetics: "One Wafer, One Network Layer"
Industry experts estimate that GPT-5.6 Sol's specifications are extremely large:
- Total Parameters: ~3 trillion
- Activated Parameters: ~150 billion
- Number of Network Layers: ~70 to 90 layers
To achieve healthy inference service characteristics, OpenAI and Cerebras have adopted an extremely luxurious and shocking deployment method — deploying each neural network layer on a separate, entire Cerebras wafer.
As one netizen pointed out, by increasing pipeline stages, as long as you have enough wafers to link them, you can theoretically scale to any model size. This does not affect the Token generation speed, only potentially slightly impacting the Time To First Token (TTFT).
Architecture Restructuring by Cutting the Gordian Knot — A Forced Lightweight KV Cache
However, having a massive number of wafers is not enough. A major feature of Cerebras chip architecture is its vast amount of on-chip SRAM (Static Random-Access Memory), which is extremely fast but has precious capacity.
If OpenAI uses the traditional heavy KV cache in GPT-5.6 Sol as before, this expensive SRAM bandwidth would be instantly consumed.
This leads to the most core strategic pivot of this collaboration: model reconstruction centered on specific hardware.
Bleys Goodson pointed out that since OpenAI was deeply involved in hardware co-design, they most likely abandoned the traditional attention mechanism caching scheme and adopted a more cutting-edge lightweight design.
The most likely solutions include:
Architecture similar to DeepSeekV4: Extremely optimized cache footprint.
Hybrid SSM Design: Combining linear-time complexity models like Mamba with Transformers, completely shedding the historical burden of KV Cache.
Furthermore, well-known developer John Lam put forward an astonishing guess — decoupling Attention and FFN.
He speculated that OpenAI might be using traditional GPUs to handle attention calculations, while using massive Cerebras wafers to brute-force push the computations of the feed-forward neural network part.
This is not groundless. Netizens quickly dug up details about Cerebras's previous blog post regarding the deployment of Kimi K2.6:
Cerebras stored Kimi K2.6's original weights at 4-bit on the CS-3 system while computing at 16-bit floating point to ensure precision. Weights are distributed across multiple wafers, and activations are streamed between wafers. The all-to-all communication between layers relies entirely on the on-wafer network fabric, whose bandwidth is over 200 times that of NVLink on Nvidia NVL72! Combined with custom operators and speculative decoding, they can run trillion-parameter MoE models at speeds close to 1000 tokens/s.
Official specifications show that the revolutionary CS-3 system is not only unbeatable in speed but can also easily scale to 24 trillion parameter models on a single logical device!
As someone exclaimed: "If this is really the full version of Sol running on Cerebras, then the preset ceiling for model size has been directly shattered tonight."
The Real Trump Card — OpenAI's First Self-Developed Chip "Jalapeño"
And just before this, OpenAI officially released its first-ever self-developed chip — Jalapeño.
The arrival of this chip directly explains the deeper logic behind OpenAI's collaboration with Cerebras: By exploring on third-party top-tier inference hardware, OpenAI has thoroughly understood the key points and value of dedicated inference architectures and converted them into a controllable underlying platform of their own.
Jalapeño is one of the mildest chili peppers in Mexico. Naming it as such, OpenAI clearly indicates: This is just an appetizer.
This chip is a custom ASIC designed specifically for large model inference. From the first line drawn, every transistor was optimized for one thing only: running large models.
Surprisingly, Jalapeño not only runs OpenAI's own models, but its architecture is also compatible with industry-wide LLMs, demonstrating great platform ambition.
Moreover, the design and tape-out of this chip took only 9 months.
Behind this is an extremely powerful industry alliance:
Architecture Leadership: OpenAI personally handles the underlying architecture design.
Chip Implementation & Interconnect: Chip giant Broadcom provides powerful implementation capabilities and network interconnect technology support.
System Integration: Celestica is responsible for final board manufacturing and rack-level physical integration.
Devouring the Entire Industry Chain, OpenAI's Full-Stack Empire Ambition
Training models themselves, designing chips themselves, optimizing inference themselves, controlling deployment themselves.
Clearly, OpenAI's goal is a vast full-stack AI empire.
But OpenAI's ambition is even crazier than Apple's and Google's. They possess an unprecedented super flywheel: using AI to accelerate the construction of AI infrastructure, then using the built, stronger infrastructure to run even more powerful AI.
According to the grand blueprint announced by OpenAI, the first batch of GW-level super data centers will begin deployment from late 2026 in collaboration with core partners like Microsoft.
The total electricity consumption of a medium-sized city will be used to power the inference racks of Jalapeño and the next-generation chili chips.
Get ready. Soon, we will welcome GPT-5.6 Sol racing on Cerebras wafers at 750 Tokens/s, breaking the physical curse of parameters and inference speed.
Reference: https://x.com/bleysg/status/2073937651150029084
This article is from the WeChat public account "New Zhiyuan," author: ASI Revelation; Editor: Aeneas
