Our ambassadors made October a massive global movement for fast inference! We hosted Cafe Compute Meetups in 15 cities:
Boston • Bengaluru • Dubai • Austin • Brisbane • Melbourne • Indore • Barcelona • Mexico City • Bhopal • Seoul • London • San Francisco • Dublin utout to our incredible Cerebras Ambassadors who are growing their local AI communities around the world. 🙌
Want to bring fast inference to your community? Apply to become a Cerebras Ambassador: cerebras.ai/ambassadors
Boston • Bengaluru • Dubai • Austin • Brisbane • Melbourne • Indore • Barcelona • Mexico City • Bhopal • Seoul • London • San Francisco • Dublin utout to our incredible Cerebras Ambassadors who are growing their local AI communities around the world. 🙌
Want to bring fast inference to your community? Apply to become a Cerebras Ambassador: cerebras.ai/ambassadors
15
13
122
10,586
Breaking it down with Sean Lie, our CTO,
How we deploy Trillion-parameter frontier models on Cerebras
How we deploy Trillion-parameter frontier models on Cerebras
How we deploy multi-trillion parameter models at scale
Today, let's break down the math for how @cerebras:
1. Fits the full model into on-chip memory
2. Maps the model across multiple chips
3. Handles a high # of concurrent requests
6
6
43
4,064
Cerebras retweeted
How to deploy multi-trillion parameter models (fast):
A single Cerebras WSE-3T wafer has 44 GB of on-chip memory, and trillion-parameter frontier models need terabytes for their weights alone.
We’ll use the open source model Kimi K3, with ~3 trillion parameters, as an For this illustrative example, we use:
- A 256k assumed average request size
- 4-bit weight quantization (like theHugging Face implementation)
- An 8-bit KV cache
## Size of Each Layer
Kimi K3 has 3 trillion parameters. If all 3T parameters are stored at 4 bits each, the raw parameter payload is:
3T x 4 bits / bits per byte = 1.5 TB
divide by 93 layers gives
1500 GB / 93 = 16 GB per layer
## Number of wafers
We can fit 2 linear layers per wafer and 1 dense layer per wafer
69 linear layers / 2 = 35 wafers
24 dense layers / 1 = 24 wafers
total = 35 + 24 wafers
## How many requests can we serve
We only need to calculate KV cache requirements for dense layers.
We can calculate the remaining space for the KV cache with Total wafer storage - size of one dense layer.
44 GB - 16 GB = 28 GB
To find the number of concurrent requests supported for any particular model mapping, we first calculate how much cache a single 256k request takes up:
576 byes / 256,000 = 147 MB per equence
Then divide:
28 GB / 147 MB = 190 concurrent requests
The same approach extends to larger models: calculate the memory needed for weights and request state, distribute the model across wafers, and connect them into a pipeline. Each additional wafer contributes compute and on-chip memory, allowing us to accommodate models that exceed the capacity of any single chip.
A single Cerebras WSE-3T wafer has 44 GB of on-chip memory, and trillion-parameter frontier models need terabytes for their weights alone.
We’ll use the open source model Kimi K3, with ~3 trillion parameters, as an For this illustrative example, we use:
- A 256k assumed average request size
- 4-bit weight quantization (like theHugging Face implementation)
- An 8-bit KV cache
## Size of Each Layer
Kimi K3 has 3 trillion parameters. If all 3T parameters are stored at 4 bits each, the raw parameter payload is:
3T x 4 bits / bits per byte = 1.5 TB
divide by 93 layers gives
1500 GB / 93 = 16 GB per layer
## Number of wafers
We can fit 2 linear layers per wafer and 1 dense layer per wafer
69 linear layers / 2 = 35 wafers
24 dense layers / 1 = 24 wafers
total = 35 + 24 wafers
## How many requests can we serve
We only need to calculate KV cache requirements for dense layers.
We can calculate the remaining space for the KV cache with Total wafer storage - size of one dense layer.
44 GB - 16 GB = 28 GB
To find the number of concurrent requests supported for any particular model mapping, we first calculate how much cache a single 256k request takes up:
576 byes / 256,000 = 147 MB per equence
Then divide:
28 GB / 147 MB = 190 concurrent requests
The same approach extends to larger models: calculate the memory needed for weights and request state, distribute the model across wafers, and connect them into a pipeline. Each additional wafer contributes compute and on-chip memory, allowing us to accommodate models that exceed the capacity of any single chip.
10
18
126
18,175
Naji Jackson has spent more than 20 years supporting leaders in tech, including @alexandr_wang and @rauchg.
She joins @MilksandMatcha on @cerebras' Big Chip Club to talk about the AI tools she has tried, why fast inference matters for making AI assistants useful, and the trust n the job.
0:00 Meet Naji Jackson
1:20 Being an executive assistant to some of the most influential people in tech
3:14 The craziest thing she’s had to do as an EA
5:10 Learning preferences and anticipating needs
6:56 Testing Grok Bot, Meta Muse, and Claude Cowork
9:42 What do AI assistants still need to improve?
11:27 Why AI assistants must be fast to be useful
15:28 Can AI replace a human assistant?
16:42 Personal assistants vs. executive assistants
17:37 How to hire a personal assistant
19:47 How to reach Naji
She joins @MilksandMatcha on @cerebras' Big Chip Club to talk about the AI tools she has tried, why fast inference matters for making AI assistants useful, and the trust n the job.
0:00 Meet Naji Jackson
1:20 Being an executive assistant to some of the most influential people in tech
3:14 The craziest thing she’s had to do as an EA
5:10 Learning preferences and anticipating needs
6:56 Testing Grok Bot, Meta Muse, and Claude Cowork
9:42 What do AI assistants still need to improve?
11:27 Why AI assistants must be fast to be useful
15:28 Can AI replace a human assistant?
16:42 Personal assistants vs. executive assistants
17:37 How to hire a personal assistant
19:47 How to reach Naji
6
19
154
74,000
Cerebras retweeted
To put this in context, it helps to compare @cerebras with how GPUs run large models. GPUs face two key challenges that the wafer-scale design avoids.
#1: Weights live off-chip
On a Cerebras wafer, each layer’s weights sit in on-chip SRAM right next to the compute that uses them. Nothing has to be fetched per token, so the bandwidth bottleneck goes away.
In contrast, a GPU has only tens of MB of on-chip SRAM, so model weights are stored in off-chip HBM. To generate each token, the GPU has to stream every layer’s weights from HBM into its compute cores. As a result, decoding is usually limited by memory bandwidth, not compute.
#2: A GPU cannot fit a model layer on chip
On a Cerebras wafer, each wafer has 44 GB of on-chip memory allowing entire model layers to fit on a single chip.
In contrast, a GPU often cannot fit a single model layer in fast on-chip memory, so you have to shard the same layer about multiple GPUs in HBM (tensor parallelism). Each GPU computes part of the layer, and the GPUs then have to combine their partial results before the next layer can start. That synchronization happens several times per layer, on every token, which adds communication overhead and latency at every step.
All of this traffic leaves the chip and travels over external links and switches. A full 72-GPU Rubin NVL72 rack, for example, connects its GPUs with roughly 5,000 internal copper cables, for 260 TB/s of total NVLink bandwidth.
A single WSE-3T has 53.5 PB/s of on-wafer fabric bandwidth, more than 200x that of the entire rack. At the small batch sizes used for fast decoding, this communication can take as long as the computation itself.
#1: Weights live off-chip
On a Cerebras wafer, each layer’s weights sit in on-chip SRAM right next to the compute that uses them. Nothing has to be fetched per token, so the bandwidth bottleneck goes away.
In contrast, a GPU has only tens of MB of on-chip SRAM, so model weights are stored in off-chip HBM. To generate each token, the GPU has to stream every layer’s weights from HBM into its compute cores. As a result, decoding is usually limited by memory bandwidth, not compute.
#2: A GPU cannot fit a model layer on chip
On a Cerebras wafer, each wafer has 44 GB of on-chip memory allowing entire model layers to fit on a single chip.
In contrast, a GPU often cannot fit a single model layer in fast on-chip memory, so you have to shard the same layer about multiple GPUs in HBM (tensor parallelism). Each GPU computes part of the layer, and the GPUs then have to combine their partial results before the next layer can start. That synchronization happens several times per layer, on every token, which adds communication overhead and latency at every step.
All of this traffic leaves the chip and travels over external links and switches. A full 72-GPU Rubin NVL72 rack, for example, connects its GPUs with roughly 5,000 internal copper cables, for 260 TB/s of total NVLink bandwidth.
A single WSE-3T has 53.5 PB/s of on-wafer fabric bandwidth, more than 200x that of the entire rack. At the small batch sizes used for fast decoding, this communication can take as long as the computation itself.
10
15
130
10,189
Use your existing AWS account to subscribe to Cerebras Inference: lnkd.in/gC2StuRS
0
4
18
4,425
Cerebras retweeted
The @cerebras team booked a world champion olympic climber, Janja Garnbret, for a private climbing workshop in SF.
Full climbing gym book out as well. Drop a #bigchipenergy for an invite.
Full climbing gym book out as well. Drop a #bigchipenergy for an invite.
51
4
136
25,414
The new physics of cybersecurity makes inference speed essential. Defenders now need to detect, reason, and act so much faster than attackers.
Read more about how Cerebras is partnering with CrowdStrike to accelerate that defense loop 👇
Thank you, @gdb, for highlighting how AI-enabled cybersecurity is more critical than ever!
Read more about how Cerebras is partnering with CrowdStrike to accelerate that defense loop 👇
Thank you, @gdb, for highlighting how AI-enabled cybersecurity is more critical than ever!
OpenAI took 25% of its production engineers off every project and told them: you are just securing now. Their top model hunted in their infrastructure until it saturated (stopped finding new issues).
Will everyone else follow?
New physics of cybersecurity
@gdb / @NoPriorsPod
15
22
181
16,054
Your AI agent has a to-do list. Waiting shouldn’t be the longest item.
Cerebras’ Andy Hock joins theCUBE to discuss why inference speed matters for reasoning models and AI agents.
Where would less waiting make the biggest difference?
Watch the full conversation with Andy Hock and theCUBE:
youtube.com/watch?v=H2Gnro…
Cerebras’ Andy Hock joins theCUBE to discuss why inference speed matters for reasoning models and AI agents.
Where would less waiting make the biggest difference?
Watch the full conversation with Andy Hock and theCUBE:
youtube.com/watch?v=H2Gnro…
4
15
78
9,820
Cerebras retweeted
AI is raising the stakes in cybersecurity.
If attacks are fast, defense must be faster.
Speed is becoming a security primitive.
If attacks are fast, defense must be faster.
Speed is becoming a security primitive.
6 days and $40,000 of compute with GPT-5.6 Sol to find a critical vulnerability in Chrome, claiming Google’s first full-chain reward in 2026 for a vulnerability that would affect an estimated 1 billion users.
Highlights the opportunity for defenders to utilize these tools to secure their software to unprecedented levels:
9
24
167
9,643
AI agents rely on a good harness.
Faster inference gives a harness room to inspect results and recover sooner, while you steer the work. @MilksandMatcha breaks down the loop.
Faster inference gives a harness room to inspect results and recover sooner, while you steer the work. @MilksandMatcha breaks down the loop.
11
17
113
137,662
Cerebras retweeted
If the cyber attack is fast, the defense must be faster. By removing inference as a bottleneck, Cerebras helps defenders act before attackers can advance. x.com/i/article/2102…
15
37
223
44,760
Cerebras retweeted
How do you SCALE FAST in this explosive moment of growth?
That was the SF Tech Week conversation I joined at @genspark_ai Spark House alongside @sang_wen @zacharylipton @shengao @MarkMcQuade.
Whether building agents, models, or AI native experiences, speed matters.
Speed of kens.
Speed to market.
At @Cerebras, we think about this all day, everyday.
Another protip: hire your marketers faster than you think! 😉
@sang_wen tysm for bringing us together.
#SFTechWeek @Techweek_
That was the SF Tech Week conversation I joined at @genspark_ai Spark House alongside @sang_wen @zacharylipton @shengao @MarkMcQuade.
Whether building agents, models, or AI native experiences, speed matters.
Speed of kens.
Speed to market.
At @Cerebras, we think about this all day, everyday.
Another protip: hire your marketers faster than you think! 😉
@sang_wen tysm for bringing us together.
#SFTechWeek @Techweek_
3
3
30
5,185