How Cerebras' Wafer-Scale Engine Works and Why It Rattles Nvidia

Photo by TECNIC Bioprocess Solutions on Unsplash

Most coverage frames Cerebras as the company that built a really big chip. That framing misses the point entirely. To understand how Cerebras wafer-scale chip works, start with what large language models actually do when they answer you: they shuttle tens of gigabytes of model weights across a memory bus, billions of times per second, to feed a relatively small amount of arithmetic. The chip is not big for spectacle. It is big because the memory and the math sit on the same piece of silicon, and that single design choice attacks the exact bottleneck that defines modern inference.

The stakes became impossible to ignore on May 14, 2026, when Cerebras went public on Nasdaq under ticker CBRS, pricing 30 million shares at $185 and raising $5.55 billion in the largest U.S. tech IPO since Uber in 2019. Shares closed Day 1 up 68% at $331.07 before giving back roughly 10% the following session.

The Misconception That Sells GPU Clusters

When people picture an AI accelerator, they picture a card. A rectangle of silicon a little smaller than a postage stamp, fed by stacks of high-bandwidth memory glued to its edges, mounted on a PCB, slotted into a server, then networked to thousands of identical cards through switches and cables that cost almost as much as the cards themselves.

Cerebras throws that diagram in the bin. The Wafer-Scale Engine 3 is the wafer. According to a breakdown of Cerebras' IPO filing, the WSE-3 is fabricated on TSMC's 5nm process, measures 46,225 square millimeters, packs 4 trillion transistors and 900,000 AI-optimized cores, and carries 44 GB of on-chip SRAM with 21 petabytes per second of memory bandwidth. The same source notes the die is roughly 57 times larger than Nvidia's H100 die at about 814 mm².

That bandwidth figure is the one to circle. The H100's HBM3 stack delivers around 3.35 TB/s, which one published analysis of Cerebras' architecture frames as roughly four orders of magnitude less than what the WSE-3 moves between its cores and its own SRAM. Four orders of magnitude is not a rounding error. It is the difference between a garden hose and a river.

The Spec That Predicts Inference Latency Better Than FLOPS

For years, the industry sold accelerators on peak FLOPS. The number on the marketing slide. Inference workloads have quietly invalidated that framing, because generating each new token in an LLM is overwhelmingly a memory operation, not an arithmetic one. The same analysis linked above puts it plainly: the bottleneck is moving model weights from memory to arithmetic units fast enough, and Cerebras sidesteps the problem because memory and compute live on the same piece of silicon.

Think of an LLM doing inference as a chef cooking from a 70-gigabyte recipe book. A GPU keeps the book on a shelf across the kitchen and runs back to fetch each ingredient. The WSE-3 prints the entire book on the countertop. The cooking step was never the slow part.

This is why peak performance numbers matter less than you might expect. The WSE-3 hits 125 petaFLOPS in FP16 with sparsity, per the same filing breakdown, but the more consequential figure for token-generation latency is that 21 PB/s of bandwidth feeding the arithmetic without ever crossing an off-chip interconnect. Sparsity is its own quiet advantage: The Register notes that Nvidia added sparsity support back in its Ampere generation but limited it to a 2:4 ratio, a narrower implementation than what Cerebras supports.

Cerebras WSE-3 vs Nvidia H100, in Numbers That Actually Matter

The direct comparison the marketing departments avoid:

• Die area: 46,225 mm² versus roughly 814 mm² for the H100, per the Cerebras filing analysis.
• Transistors: 4 trillion on a single WSE-3 wafer, against the H100's 80 billion on its monolithic die.
• On-chip memory: 44 GB of SRAM on the WSE-3. The H100 carries a few tens of megabytes of SRAM and relies on 80 GB of HBM3 sitting off-die.
• Memory bandwidth: 21 PB/s on-chip for the WSE-3 against 3.35 TB/s of HBM3 for the H100, as documented in independent analysis.
• Cores: 900,000 AI-optimized cores per wafer, versus the H100's 14,592 CUDA cores plus 456 fourth-gen Tensor Cores.

For what is Cerebras Systems actually selling at the rack level, the answer is the CS-3 system. The filing breakdown lists Cerebras CS-3 specs at roughly 25 kW of power draw per system, with the company claiming it can replace hundreds of GPUs for specific inference workloads. A claim of that shape needs a footnote, and the analysis in Medium supplies it: wafer yield economics are not publicly disclosed, so cost per functional TFLOP before system integration is unverified, which makes third-party TCO comparisons assertions rather than facts.

The Hidden Ceiling Nobody Puts on a Slide

Here is the trade-off Cerebras decks tend to skip. Putting all your memory on the wafer means you live within whatever SRAM fits on the wafer. Today that is 44 GB. A trillion-parameter model does not fit on one WSE-3, and the math gets ugly fast.

The Register's IPO writeup puts a number on it: serving a trillion-parameter model like Kimi K2 requires between 12 and 48 WSE-3 accelerators depending on weight storage format and pruning. That is the practical ceiling of the current generation, and it is the reason the same report describes the WSE-3 as getting long in the tooth, with a WSE-4 expected to push floating-point performance at lower precisions like FP8 and FP4, possibly using TSMC 3D chip-stacking to lift SRAM beyond 44 GB.

Lower precision is the real lever. FP4 quadruples the effective parameter capacity of the same SRAM versus FP16. Stacked SRAM compounds it. Whether Cerebras ships WSE-4 in 2026 or 2027 determines whether the trillion-parameter ceiling moves down to one or two systems, or stays at twelve.

There is also a supply problem the prospectus admits. Investing.com's reporting on the S-1 flags that Cerebras has no formalized long-term supply or allocation commitment from TSMC and buys wafers on a purchase order basis, which makes its $24.6 billion revenue backlog contingent on foundry access the company does not control. Nvidia, by contrast, has years of negotiated capacity.

What 565 Lines of Code Tells You About the Software Story

GPU clusters are not slow only because of memory bandwidth. They are slow to program. Pipeline parallelism, tensor parallelism, ZeRO sharding, gradient checkpointing, communication-aware schedulers: getting a 175-billion-parameter model trained across thousands of H100s is an exercise in distributed systems engineering as much as machine learning.

The Medium analysis cites that the same 175B-parameter model can reportedly be trained on a Cerebras system in 565 lines of code. The reason is structural: when the model fits on the chip, the orchestration vanishes. There is no cluster to coordinate. There is one device.

That is a developer-productivity moat as much as a performance one. It is also the spec Nvidia cannot copy without changing its silicon strategy, which is why Nvidia bought its way into the conversation instead. According to Investing.com, Nvidia acquired inference startup Groq's assets for $20 billion in December 2025, giving it an SRAM-packed inference platform and narrowing Cerebras' architectural moat in the process.

The Inference Bet, and Who Is Paying For It

For anyone trying to get the Cerebras IPO AI chip explained in one paragraph: this is a company that bet, in 2016, that inference would eventually dwarf training as the dominant AI workload, and built silicon optimized for the bandwidth-bound, latency-sensitive part of that pie. CNBC reports the company's specialty is inference, the phase where AI models respond and interact directly with users, not just training.

The customers validate the bet. Investing.com's prospectus breakdown describes the OpenAI Master Relationship Agreement as a commitment of 750 megawatts of Cerebras-backed low-latency inference, valued at over $20 billion in the S-1, with OpenAI specifically deploying Cerebras for a code-generation model where latency is non-negotiable. The same report describes AWS's March 2026 move to deploy CS-3 systems inside Amazon data centers and expose them through Amazon Bedrock, pairing Trainium for prefill with Cerebras for decode and targeting 5x more high-speed token capacity in the same hardware footprint.

That prefill-versus-decode split is the part most readers will miss. Prefill is throughput-bound, decode is latency-bound, and AWS's architecture acknowledges that the two phases of inference want different silicon. Cerebras wins decode. Nvidia and AWS's own chips win prefill. The cluster does both.

For more context on the wider economics, our AI Infrastructure articles cover the data-center side of this shift, and developers looking at the practical layer can see how to run low-latency inference on the Cerebras CS-3 API.

What the Founders Got Right in 2016

The filing analysis lays out the lineage: Cerebras was founded in 2016 by CEO Andrew Feldman and Chief Architect Michael James, who previously co-founded SeaMicro and sold it to AMD for $334 million in 2012. The WSE-1 arrived in 2019, the WSE-2 in 2021, and the WSE-3 in March 2024, with each generation roughly doubling performance inside the same power envelope. The company has raised over $4 billion privately, including a $1 billion Series H in February 2026 at a $23 billion valuation.

That 2016 founding date is the part that matters. Cerebras committed to wafer-scale years before ChatGPT made inference latency a board-level concern. The architecture was a bet on memory bandwidth as the constraint, made when the constraint was still compute.

CNBC's IPO recap priced the post-pop market cap at roughly $95 billion. By any sensible multiple of disclosed revenue, that valuation is paying for the next five years of inference demand on hardware that has not been refreshed in two years. The market is pricing WSE-4 and WSE-5 as much as it is pricing WSE-3.

Three Things to Think Differently About Now

If you take one shift in mental model away from this, make it this: stop comparing AI accelerators by FLOPS. Compare them by bytes moved per second per dollar, and by how many parameters fit in the fastest tier of memory. That single reframing collapses most of the marketing material the industry produces.

Second, the moat is not the wafer. The moat is what the wafer makes redundant: the InfiniBand fabric, the NVLink switches, the multi-month effort of pipeline-parallel model engineering. When Nvidia bought Groq's assets, it bought a path to that same redundancy at a smaller scale. The next eighteen months are about whether Cerebras' lead in single-system inference capacity translates into a durable software lock-in before Nvidia's SRAM-heavy offering matures.

Third, the supply story matters more than the spec sheet. A company without a multi-year TSMC allocation, selling into a market where every competitor is fighting for the same 5nm and 3nm capacity, is a company whose backlog is partly aspirational. Read the next earnings call for the foundry commitments, not the FLOPS.