It is equally if not more likely that the great filter has more to do with how a civilization *avoids* certain technologies that would result in wireheading, nuclear bombs, and other societal-ruining technologies (which unequivocally exist) rather than a lack of tech-tree depth many e/accs heads.
nickbostrom.com/papers/vulnera…
nickbostrom.com/papers/vulnera…
2
0
14
807
Anthropic is going to win because they are a cult who believes and belief is very powerful
10
17
366
6,732
Is this a neo-SA lengthy article + fund launch?
The benefits of abundant intelligence will accrue far outside of San Fransisco.
Spent the last year writing about capital, sovereignty, and the coming reordering of economic power.
THE COMMANDING HEIGHTS
Commanding-Heights.com
2
0
12
1,279
Please read this, think about the broader picture, and tell me from an *economic* perspective how local wins in the next 5 years.
I'm dumping the majority of my Intel shares. This was genuinely one of the saddest earnings calls.
I had SO much hope for this company. I genuinely believed Intel was finally turning things around under Lip-Bu Tan and to be fair, they've made some real progress. But after earnings call, I honestly think Intel messed up BAD. And what makes this even more frustrating is that Intel has one of the biggest opportunities in the semiconductor industry sitting right in front of it, yet it cannot fully take advantage of it.
First, the guidance was absolutely terrible. Intel reported $16.5 billion in Q3 revenue and adjusted EPS of $0.52, which were actually pretty solid results. But then management guided for just $15 billion to $16 billion in Q4 revenue, compared to Wall Street's expectations of around $17.1 billion. They also expect adjusted EPS to fall to $0.25, compared to expectations of roughly $0.43. Even worse, gross margins are expected to fall from 44.1% to just 38% in a single quarter. That's a massive decline, especially for a company that's supposed to be improving its profitability.
But the biggest problem is what management said about 2027. Intel's PC business is getting destroyed by the ongoing memory shortage. Companies like Samsung, SK Hynix, and Micron are benefiting from enormous AI demand, but that same demand is making memory more expensive and harder to find for traditional computers.
And when memory becomes more expensive, the entire computer becomes more expensive. That means fewer people and businesses are willing to upgrade their PCs, which directly hurts Intel. Management now expects the overall PC market to shrink by around 12% in 2026 and another 10% in 2027. Intel also expects its own client revenue to fall more than 20% in Q4 alone. What really caught my attention was when management basically admitted that they don't expect the memory situation to improve in 2027. So this isn't just a temporary problem that disappears next quarter. It could continue hurting Intel for another year or longer.
Now, here's what frustrates me the MOST. Intel's data center business is absolutely booming, but they cannot manufacture enough chips to meet demand. Data Center and AI revenue grew nearly 80% year over year to $7.3 billion. Management expects industry server CPU shipments to grow more than 30% in both 2026 and 2027, and Lip-Bu Tan even believes the server CPU market could grow from around $40 billion today to more than $200 billion over the next five years because of AI agents. That's an unbelievable opportunity. But Intel is dealing with manufacturing capacity limitations, production bottlenecks, and shortages of the materials needed to package its chips. In other words, customers WANT to buy more Intel server CPUs, but Intel cannot produce enough of them.
And management made it pretty clear that these supply problems will continue into the first half of 2027, with meaningful improvements not expected until the second half. This is what bothers me more than anything. We're experiencing one of the strongest periods of CPU demand in years, and Intel is leaving billions of dollars in potential sales on the table while competitors fight for market share. Then there's the tragic accident at Intel's factory in Israel, which forced production to shut down for approximately two weeks. Management expects this to reduce Q4 revenue by around $300 million, with a similar impact potentially extending into Q1. It also accounts for about two percentage points of the expected Q4 gross margin decline. Obviously, this was an unfortunate situation, but it creates even more problems for a company that was already struggling to produce enough chips.
And don't even get me started on the foundry business. This was supposed to be one of the biggest reasons to invest in Intel. The idea was that Intel could become a serious competitor to TSMC by manufacturing chips for other companies. And yes, Intel is making progress. Its 18A manufacturing process is improving, production costs are coming down, and management sounds more confident about 14A but here's the reality. Intel's foundry business still lost $1.9 BILLION in Q3. External foundry revenue was only $411 million, and part of that came from a one time customer program. Management keeps talking about potential customers, improving yields, and future opportunities. But where are the major external customer commitments that will actually make this business profitable? I'm not saying they won't come. I'm saying investors have already been waiting years for this turnaround, and we're still talking about what MIGHT happen in 2027 and 2028.
And then came another comment that really disappointed me. Intel doesn't expect to generate positive free cash flow in 2027. The company already expects to spend around $20 billion on capital expenditures in 2026, and management said spending will increase significantly next year. So we're looking at another year of massive investments, continued foundry losses, a declining PC market, and potentially negative free cash flow. And remember, Intel raised $23 billion by issuing additional shares in August, meaning existing shareholders also have to deal with dilution. At some point, investors need to ask when all this spending is actually going to translate into sustainable profits.
Now, I want to be fair because there ARE positives here. Intel's server CPU demand is incredibly strong. Its manufacturing technology is improving. Granite Rapids is becoming more profitable, its custom silicon business is growing rapidly, and its upcoming products like Diamond Rapids and Nova Lake could help it regain market share. I also think the long term opportunity in AI infrastructure, advanced packaging, and foundry services is enormous. But here's the issue. Almost every major positive Intel talks about is something that could happen in the FUTURE. More manufacturing capacity in the second half of 2027. Potential foundry customers. A stronger product roadmap. Better profitability. Eventually positive free cash flow.
Meanwhile, all the problems are happening RIGHT NOW. The PC business is declining right now. Manufacturing constraints are hurting sales right now. Foundry is losing billions right now. And margins are expected to fall significantly next quarter. The worst part is that Intel doesn't even have a demand problem in its server business. It has an execution problem. And that's arguably even more frustrating because the opportunity is already there.
I understand that rebuilding a company like Intel takes years, and I still respect what Lip-Bu Tan is trying to accomplish. But after the massive rally in this stock, investors were expecting Intel to finally start delivering meaningful results, not another year of excuses and promises. I was willing to be patient with Intel because I believed the turnaround was gaining momentum. But this earnings call genuinely changed my perspective.
I'm not selling because I suddenly believe Intel is a terrible company or that its technology has no future. I'm selling because I no longer think the risk and reward are nearly as attractive as they were before. I'm dumping the majority of my position, but I'll likely keep a smaller stake because I still believe there's long-term potential here. I really wanted Intel to succeed. I still do. But wanting a company to succeed and believing its stock is a good investment at the current valuation are two completely different things.
And after listening to management today, I just cannot justify maintaining the same level of exposure. Intel had the opportunity to prove that its turnaround was finally working. Instead, management gave investors even more reasons to question it. This was genuinely disappointing.
2
0
22
4,033
Unfortunately, a great point by Sam. If we really are in such a memory crunch, in what world does it make sense to have all the world’s DDR distributed across local devices being used only 5% of the day? Inference economics would be much more favorable if memory costs could be amortized over higher utilization.
sam: local inference is a terrible allocation of scarce resources
“a lot of people want local inference, and somehow they assume that the problem is solved now because we have open-source models, we're going to be able to run them on our computers, and so we're not going to need ta centre
but from a global allocation perspective, actually it's quite terrible to have everybody, like, have a bunch of the world's limited supply of DRAM sitting in their own computer and being used five percent of the time
you want maximum utilisation in the cloud if you're trying to deal with this very supply-constrained environment
somehow many people's intuition is the opposite of that”
228
10
211
42,378
A lot of people are replying with political arguments while I am making an economic argument. I want local to succeed, but the market determines all.
19
0
21
4,022
Remember when Elon said "A few bit flips due to radiation won't change the model that much"
if you have a qwen3.5 4b around, please try steering neurons L14 N8949 +1 and L13 N7956 -1
now you have evil qwen
recipe: dose × median(‖resid‖ leaving layer L, tokens 1:) / ‖W_down[:, n]‖` at the down_proj at every input
0
1
24
1,624
The great irony here is that Cerebras has previously criticized Nvidia Blackwell packaging for CTE mismatch between the dies, interposer, and organic substrate while claiming that their monolithic WSE would solve such thermal expansion issues through the absence of an interposer ckaging."
Yet now Cerebras (according to this research) has to run background, non-functional workloads to keep its wafers warm and avoid damaging thermal cycles.
Even with a uniform CTE, a wafer this large is still susceptible to mechanical stress from spatial temperature gradients (imagine one corner of the chip running significantly hotter, and thus expanding relative to the rest).
Even funnier given that their S-1 claims "thermal expansion-tolerant packaging" without mentioning any requirement to artificially keep wafers warm.
sec.gov/Archives/edgar…
Yet now Cerebras (according to this research) has to run background, non-functional workloads to keep its wafers warm and avoid damaging thermal cycles.
Even with a uniform CTE, a wafer this large is still susceptible to mechanical stress from spatial temperature gradients (imagine one corner of the chip running significantly hotter, and thus expanding relative to the rest).
Even funnier given that their S-1 claims "thermal expansion-tolerant packaging" without mentioning any requirement to artificially keep wafers warm.
sec.gov/Archives/edgar…
NEW: We are short Cerebras Systems $CBRS. Cerebras claims to have solved wafer-scale chips. We believe otherwise: $CBRS wafers break down after just ~2 years in use - a lifespan that undermines the entire business. See full report and disclosures on our website, link in bio.
0
2
10
1,539
It is such a funny twist of fate that OpenAI, led by an ex-YC product guy, is putting out groundbreaking math/physics/cs research while Anthropic, led by a physics PhD, is primary putting out a coding product.
Announcing three major scientific discoveries, one each in mathematics (harmonic analysis), computer science (complexity theory), and physics (quantum field theory).
AI starting to help us improve our understanding of the universe! openai.com/index/building…
37
49
1,617
47,277
It is also interesting to put this in the context of modern education. Perhaps one reason Europe is so obsessed with Master's Degrees is because being a student is an alibi for not providing economic utility.
When your economy is terrible and it is impossible to get a job or provide any actual value because of bureaucracy, one may lean on perpetual 'Education' and credentialing as an excuse for why they are unemployed. In many cultures, 'student' is technically a job and thus an excuse to avoid the increasingly nightmarish recruiting process.
I see this a lot here in the US as well. Friends who have recently graduated undergrad have applied to 1000+ jobs with no luck, then 'give up' and just enroll in a Master's program which their Boomer parents gleefully pay for because they think that Master's Degrees still have any actual value. Similar idea to how an increase in Law School applications is a recession indicator.
Also, being a perpetual student is quite attractive especially to those seeking high status and low risk. It lets one 'be a kid' well into their 20s. It used to be that you were expected to start providing economic utility at 18; then we delayed that to 22/23 with mass college enrollment, now development is being arrested far into Zoomers 20s under the pretext of 'education' and 'credentials'. Perhaps another reason for the TFR dropping off a cliff.
When your economy is terrible and it is impossible to get a job or provide any actual value because of bureaucracy, one may lean on perpetual 'Education' and credentialing as an excuse for why they are unemployed. In many cultures, 'student' is technically a job and thus an excuse to avoid the increasingly nightmarish recruiting process.
I see this a lot here in the US as well. Friends who have recently graduated undergrad have applied to 1000+ jobs with no luck, then 'give up' and just enroll in a Master's program which their Boomer parents gleefully pay for because they think that Master's Degrees still have any actual value. Similar idea to how an increase in Law School applications is a recession indicator.
Also, being a perpetual student is quite attractive especially to those seeking high status and low risk. It lets one 'be a kid' well into their 20s. It used to be that you were expected to start providing economic utility at 18; then we delayed that to 22/23 with mass college enrollment, now development is being arrested far into Zoomers 20s under the pretext of 'education' and 'credentials'. Perhaps another reason for the TFR dropping off a cliff.
I'm quite worried about the future we are inevitably heading towards where the threshold for economic utility rises above what ~99% of the human population can meet. Contrasting Dario and Sam on this issue wrt AI is so funny to watch.
Dario is a hyper-autist who, for all his faults, I think genuinely speaks his mind. He sees the tsunami coming. Software engineering may very well be gone in the next few years along with many other computer-facing careers (of course assuming one is running an actual company and not a UBI corporation, which most of Big Tech actually is).
He is trying to warn the normies that this is something that is actually coming, and all the screaming and crying about AI and datacenters will at this point only temporarily delay the inevitable. It is easy to dunk on Dario for peddling this doomerism, but can anyone refute his actual worry?
Sam, on the other hand, is hyper-Machiavellian. He has the social intelligence to recognize that telling the normies AI is gonna take all their jobs (whether it actually will or not) is bad for his company. So he has to preach this blatant lie that he clearly doesn't believe that 'AI is gonna create more jobs'. Yeah no it won't (with the exception of human data harvesters).
For what it's worth, it seems like one of the main reasons the economy has still been fairly agnostic to 'AI efficiency improvements' is because most jobs are fake anyways. People are not becoming 'More Efficient With AI™' because there was never anything for them to make more efficient in the first place (other than maybe looking busy).
To what extent does AI force society to confront how much economic activity was never particularly productive in the first place?
So what happens if we become more honest with ourselves about the actual utility of most jobs, especially when a computer can do them extremely easily and cheaply? What happens when 99%+ of the human population has little to no economic utility?
I think this social/cultural revolution (downstream of the AI revolution) is affecting young people most of all. I know kids who graduated from HYPSM+ who literally cannot get a job. Look at any honest news article and *minimum* 50% of new college grads are un/underemployed. I think this is a sign of things to come and how we respond to millions of people having less economic utility than the computer will likely become the defining challenge of our generation. x.com/kirillk_web3/s…
3
1
52
7,211
I'm quite worried about the future we are inevitably heading towards where the threshold for economic utility rises above what ~99% of the human population can meet. Contrasting Dario and Sam on this issue wrt AI is so funny to watch.
Dario is a hyper-autist who, for all his faults, I think genuinely speaks his mind. He sees the tsunami coming. Software engineering may very well be gone in the next few years along with many other computer-facing careers (of course assuming one is running an actual company and not a UBI corporation, which most of Big Tech actually is).
He is trying to warn the normies that this is something that is actually coming, and all the screaming and crying about AI and datacenters will at this point only temporarily delay the inevitable. It is easy to dunk on Dario for peddling this doomerism, but can anyone refute his actual worry?
Sam, on the other hand, is hyper-Machiavellian. He has the social intelligence to recognize that telling the normies AI is gonna take all their jobs (whether it actually will or not) is bad for his company. So he has to preach this blatant lie that he clearly doesn't believe that 'AI is gonna create more jobs'. Yeah no it won't (with the exception of human data harvesters).
For what it's worth, it seems like one of the main reasons the economy has still been fairly agnostic to 'AI efficiency improvements' is because most jobs are fake anyways. People are not becoming 'More Efficient With AI™' because there was never anything for them to make more efficient in the first place (other than maybe looking busy).
To what extent does AI force society to confront how much economic activity was never particularly productive in the first place?
So what happens if we become more honest with ourselves about the actual utility of most jobs, especially when a computer can do them extremely easily and cheaply? What happens when 99%+ of the human population has little to no economic utility?
I think this social/cultural revolution (downstream of the AI revolution) is affecting young people most of all. I know kids who graduated from HYPSM+ who literally cannot get a job. Look at any honest news article and *minimum* 50% of new college grads are un/underemployed. I think this is a sign of things to come and how we respond to millions of people having less economic utility than the computer will likely become the defining challenge of our generation. x.com/kirillk_web3/s…
Dario is a hyper-autist who, for all his faults, I think genuinely speaks his mind. He sees the tsunami coming. Software engineering may very well be gone in the next few years along with many other computer-facing careers (of course assuming one is running an actual company and not a UBI corporation, which most of Big Tech actually is).
He is trying to warn the normies that this is something that is actually coming, and all the screaming and crying about AI and datacenters will at this point only temporarily delay the inevitable. It is easy to dunk on Dario for peddling this doomerism, but can anyone refute his actual worry?
Sam, on the other hand, is hyper-Machiavellian. He has the social intelligence to recognize that telling the normies AI is gonna take all their jobs (whether it actually will or not) is bad for his company. So he has to preach this blatant lie that he clearly doesn't believe that 'AI is gonna create more jobs'. Yeah no it won't (with the exception of human data harvesters).
For what it's worth, it seems like one of the main reasons the economy has still been fairly agnostic to 'AI efficiency improvements' is because most jobs are fake anyways. People are not becoming 'More Efficient With AI™' because there was never anything for them to make more efficient in the first place (other than maybe looking busy).
To what extent does AI force society to confront how much economic activity was never particularly productive in the first place?
So what happens if we become more honest with ourselves about the actual utility of most jobs, especially when a computer can do them extremely easily and cheaply? What happens when 99%+ of the human population has little to no economic utility?
I think this social/cultural revolution (downstream of the AI revolution) is affecting young people most of all. I know kids who graduated from HYPSM+ who literally cannot get a job. Look at any honest news article and *minimum* 50% of new college grads are un/underemployed. I think this is a sign of things to come and how we respond to millions of people having less economic utility than the computer will likely become the defining challenge of our generation. x.com/kirillk_web3/s…
40
27
324
80,558
> "Zooming back to today, inference is growing incredibly fast. According to a 2026 Goldman Sachs report, demand for tokens will increase by 24x from 2026 to 2030, yet available power will only increase by about 2x."
I don't think many people appreciate how hard it will be for the US to grow its power output at this rate. If such speculated inference growth is correct, the *only* answer is going to be ultra-low power specialized inference chips (likely paired with GPUs in heterogeneous disaggregation).
The most important metric to watch over the next few years is tok/s/watt. A substantial share of inference energy is spent moving data rather than performing arithmetic (between HBM -> SRAM). Interestingly, moving data isn't just a bandwidth bottleneck that limits decode throughput; it's also a major energy constraint that limits inference efficiency.
'Retrieving' a bit from an SRAM chip (Groq or Cerebras) uses ~10-20x less energy than a GPU fetching the same bit from HBM. SRAM chips also have far lower latency because they do not have as much data movement overhead and can just whip the residual around on-chip while the weights stay local.
So for decode-heavy inference, heterogeneous disaggregation makes obvious sense. I think the main issue, as Zain mentions here, is specialization vs generalization. Neoclouds love GPUs because they are great for training and good enough for inference, whereas this sorta disagg is primarily beneficial for mem-bandwidth bound decode.
I think the main issue currently is that clouds are still skeptical of token (decode) growth in the coming years and are hesitant to commit to massive inference-only clusters. Regardless, if token usage continues to significantly outpace the grid, I think specialized chips will be the primary means of inference for a long time.
I don't think many people appreciate how hard it will be for the US to grow its power output at this rate. If such speculated inference growth is correct, the *only* answer is going to be ultra-low power specialized inference chips (likely paired with GPUs in heterogeneous disaggregation).
The most important metric to watch over the next few years is tok/s/watt. A substantial share of inference energy is spent moving data rather than performing arithmetic (between HBM -> SRAM). Interestingly, moving data isn't just a bandwidth bottleneck that limits decode throughput; it's also a major energy constraint that limits inference efficiency.
'Retrieving' a bit from an SRAM chip (Groq or Cerebras) uses ~10-20x less energy than a GPU fetching the same bit from HBM. SRAM chips also have far lower latency because they do not have as much data movement overhead and can just whip the residual around on-chip while the weights stay local.
So for decode-heavy inference, heterogeneous disaggregation makes obvious sense. I think the main issue, as Zain mentions here, is specialization vs generalization. Neoclouds love GPUs because they are great for training and good enough for inference, whereas this sorta disagg is primarily beneficial for mem-bandwidth bound decode.
I think the main issue currently is that clouds are still skeptical of token (decode) growth in the coming years and are hesitant to commit to massive inference-only clusters. Regardless, if token usage continues to significantly outpace the grid, I think specialized chips will be the primary means of inference for a long time.
0
2
13
1,657
It has been interesting to watch San Francisco as it has been stripped of nearly all of its original countercultural garments (distrust/apathy towards The Cathedral, high-openness, suspicion of credentialism, original thought, etc.) and has donned a new zeitgeist that is the absolute antithesis of its former ethos.
Everywhere you look resembles third-world, low-trust, social hierarchy jostling . "How much did you raise bro? Did you get the Sequoia check? You don't work at a Tier 1 lab? You didn't... didn't go to Stanford?".
The city that was once great because of its disgust with these sorts of people has now been corrupted by them as they have min-maxed Machiavellian social dynamics to accumulate power in the valley like XP points. Power which is now completely divorced from any genuine talent, skill, or benevolence. All that matters is that you are able to Scam a16z out of $15M (@cluely) because you are a poaster and you are anointed with SF Clout™
This infestation is so deep that even those who rightfully call it out are still willful participants in the exact culture they so detest (check her bio).
Everywhere you look resembles third-world, low-trust, social hierarchy jostling . "How much did you raise bro? Did you get the Sequoia check? You don't work at a Tier 1 lab? You didn't... didn't go to Stanford?".
The city that was once great because of its disgust with these sorts of people has now been corrupted by them as they have min-maxed Machiavellian social dynamics to accumulate power in the valley like XP points. Power which is now completely divorced from any genuine talent, skill, or benevolence. All that matters is that you are able to Scam a16z out of $15M (@cluely) because you are a poaster and you are anointed with SF Clout™
This infestation is so deep that even those who rightfully call it out are still willful participants in the exact culture they so detest (check her bio).
made me physically disgusted reading this
>literally the only reason why you care about a person you have not met is because they've done something publicly approved
not only is it not true, this kind of mindset is so inhumane. sad for him but also the fact that lots of tech bros think like this to some degree. reject treating people transactionally.
44
52
1,143
94,098
Clear indictment OpenAI is compute constrained and is gonna need to rent GWs from SpaceX
Wow the new roadster looks quite amazing, good job @elonmusk
1
2
48
2,088
Apple knows what they are doing. Especially with Tim Cook’s diplomatic skills, if they can lock up CXMT DRAM and make a SoTA NPU/mini-TPU on Mac Studios they might end up capturing the majority of AI revenue in the limit if models move local
Muse Spark Open -- weights releasing today -- runs great on new Mac Studios thanks to work with @johnternus and Apple.
2
1
33
1,563
Sports are gonna be weird in 50 years when all the athletes import motor skills that spent 100k years in an RL gym through BCIs
NEW SIM RESEARCH: Humanoid learns basketball skills!
Called Learning to Ball, it involves a physics-simulated humanoid character that plays basketball inside IsaacGym.
It trains seven primitive skills (dribble, gather, shoot, pass, catch, rebound, and a combined adversarial imitation, then composes them into long-horizon plays.
It replaces hard skill-chaining and one-hot mixture-of-experts, which break when subtask policies share few states.
The interesting core of this project is the soft router, blending all primitives every timestep rather than switching to one.
Turning off the high-level router drops shooting accuracy significantly, and it beats a one-hot hard router precisely on the ill-defined in-between states where a switch would stutter. Continuous weighting keeps transitions smooth through states that do not belong to a single primitive.
1
0
24
1,495
Dwarkesh kicked Lex out of the technical podcast-sphere so now he has to try his hand at Clavicular-esque kick streams
Jensen reveals where his childhood love of night markets comes from
"My mom and dad loved it. We would come to the night market. When we go out to dinner, that's the definition of going out to dinner. You don't go to a restaurant. We didn't have money like that. We'd just go to t at these tables."
"I was looking for a restroom so I wandered off. I walked up to this guy, this cook. I was about to ask him if there was a restroom somewhere. He was cleaning his knife, and my face was right next to his knife. It cut me right across my face. I was almost five. My dad picked me up and ran me to the hospital."
"I always loved the night market after that. I always had a fond memory. The food was so good. I loved the place."
"I loved everything. I loved all the people. Everybody's working hard for a living. They're working so hard, and they still want to share a little something with you."
"They don't have much money. Their profit margin must be a dime each. They gave me all their profit, just to be kind. It's the Taiwanese way."
27
31
1,653
103,487
Nature already concluded disaggregation is optimal for biological computing, why should silicon be any different?
845 tokens/sec per user. ~50% lower latency.
d-Matrix Corsair + NVIDIA H100 are redefining AI inference with attention-FFN disaggregation.
Faster inference. Smarter compute.
Read more: d-matrix.ai/accelerating-a…
0
0
9
860
Prediction
ML silicon started off unified (single GPUs/TPUs)
Then grew into larger sharded clusters (NVL72s)
And is now beginning to grow into more heterogeneous, disaggregated structures. Inference providers are realizing they can get better tok/watt using GPUs for prefill/FFNs and WSE/Raptors for decode/attention layers and CPUs for agent orchestration.
I think that this branching will continue as we get independent chips for projections, FFNs, attention, normalizations, tool-use, etc. A dataflow-based rack entirely designed around the interconnect and data-locality to minimize power. I think we will look back in 5 years on the idea of using the same cores for both FFNs and attention layers as absurd.
The science of silicon is extremely advanced, but the engineering putting it to use is still in its infancy. Much of this is because we were able to lean on Moores law for so long we could afford to not worry so much about our hardware designs. Now this naivety is beginning to show itself.
arxiv.org/abs/2608.03741
ML silicon started off unified (single GPUs/TPUs)
Then grew into larger sharded clusters (NVL72s)
And is now beginning to grow into more heterogeneous, disaggregated structures. Inference providers are realizing they can get better tok/watt using GPUs for prefill/FFNs and WSE/Raptors for decode/attention layers and CPUs for agent orchestration.
I think that this branching will continue as we get independent chips for projections, FFNs, attention, normalizations, tool-use, etc. A dataflow-based rack entirely designed around the interconnect and data-locality to minimize power. I think we will look back in 5 years on the idea of using the same cores for both FFNs and attention layers as absurd.
The science of silicon is extremely advanced, but the engineering putting it to use is still in its infancy. Much of this is because we were able to lean on Moores law for so long we could afford to not worry so much about our hardware designs. Now this naivety is beginning to show itself.
arxiv.org/abs/2608.03741
8
13
125
8,868