Cool work, Albert and team!
I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data.
Language s a microcosm of this ideal, and to me it's a central problem in AI: if we can't build LLMs that learn end-to-end from bytes, how will we handle the rest of the world's data that we understand even less?
Last year we introduced hierarchical networks (H-Nets) as a first step. Our new version finds up to 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. The posts below walk through what we found and why we think it works.
1
0
5
370
Jared Quincy Davis retweeted
can't believe it's actually working
OpenAI has announced to three new discoveries in mathematics, computer science, and physics:
- Local smoothing and the Kakeya conjecture
- The Exponential Time Hypothesis is false
- Existence of quantum chromodynamics and completions of the Standard Model
22
102
2,108
67,972
Jared Quincy Davis retweeted
vLLM is all you need to reproduce DeepSeek's performance.
🚨🚨🚨 vLLM is all you need to reproduce DeepSeek’s performance.
With Wide-EP and EPLB, vLLM can match DeepInfer’s decode performance within 6% and generate $39/gpu/hr in revenue at DeepSeek’s off-peak pricing, at 200 P90 TPS on the AgentX workload.
Specifically:
• Under guration (DeepSeek-V4.1-Flash, 131K context, nextn=3), vLLM on GB200 achieves throughput within 6% of DeepInfer’s published results across batch sizes 8–256.
• On AgentX, vLLM pushes the Pareto frontier further on GB300, reaching up to 1M tokens/s/GPU and generating $38.96/gpu/hr in revenue at 196 P90 TPS, based on DeepSeek’s off-peak API pricing.
• Also, vLLM extended low-latency kernel support for DeepSeek-V4.1-Flash, achieving up to 826 TPS per user on AgentX.
1
11
141
9,823
Jared Quincy Davis retweeted
"The invention of the boat reduced the importance of swimming, but enabled the discovery of new lands."
Beautiful analogy from @ylecun about the opportunities and changes with AI, in mathematics and in science more broadly.
Beautiful analogy from @ylecun about the opportunities and changes with AI, in mathematics and in science more broadly.
35
156
1,161
54,762
Jared Quincy Davis retweeted
Everyone has been obsessed by Jev and what it can do, and they should be! Fast typed decisions >> slow text generation. It's a complete game changer. But how do you do that with all the data you have?
We just dropped 𝚊𝚒_𝚍𝚎𝚌𝚒𝚍𝚎(), which runs decision models natively on all yo. Now you can run system one decisions on all your big data, see screenshot.
databricks.com/blog/introduci…
We just dropped 𝚊𝚒_𝚍𝚎𝚌𝚒𝚍𝚎(), which runs decision models natively on all yo. Now you can run system one decisions on all your big data, see screenshot.
databricks.com/blog/introduci…
22
70
317
25,779
Jared Quincy Davis retweeted
@minchoi Colossus 1 is 150k H100, 50k H200 and 30k GB200.
Colossus 2 is 110k GB200 and 440k GB300.
Another 220k GB300 will be fully operational next week and another 220k in November. If we get lucky, yet another 220k GB300 by late December.
Colossus 2 is 110k GB200 and 440k GB300.
Another 220k GB300 will be fully operational next week and another 220k in November. If we get lucky, yet another 220k GB300 by late December.
944
1,152
15,790
2,962,584
@alexandr_wang deciding whether to load up on or trim $META holdings after reading this tweet...
0
0
1
118
Jared Quincy Davis retweeted
Quite an interesting turn in LLM diversity, if the market starts looking for specialized models that are optimized and branded for different use cases.
Should be very good for the market and open more room for non-frontier neolabs!
Should be very good for the market and open more room for non-frontier neolabs!
29
12
199
12,579
@alexandr_wang deciding whether to load up on or trim $META holdings after reading this tweet...
1
0
5
695
Jared Quincy Davis retweeted
I asked each investor for the most elite startups they'd tell an ambitious relative to join, the ones they'd join themselves if they had to join a company, and the ones they'd pick if they had to put 10% of their net worth into one company at today's price.
The list: breakoutlist.com
Breakout List, all picks (1 of 4):
1-10 people:
- Hone (@moritz_stephan, @CarloWillem, @oqbrady)
- Normal (@ansonyuu, @hudzah)
- Standard Intelligence (@G413N, @devanshpandey)
- Tacit Labs (@ninklefitz, @AmDroste)
11-25 people:
- American Terawatt (@atroyn, @rslparker, @aranibatta)
- Conduit (@clemvonstengel, @riopopper)
- Convergent (Omkar Savant, Vivek Katara, @debnilsur)
- Core Automation (@MillionInt, @_arohan_)
- Engram (@dan_biderman, @EyubogluSabri, @realJessyLin)
- Instinct (@noahrshinn)
- Keenable (@styskin, Matthias Petri)
- Lumaril (Mark Elliot, Ben Duffield)
- Neion Bio (@Dimkell, Sam Levin)
- Pangram Labs (@max_spero_, @bradley_emi)
- Quadrillion (@echinaceous)
- Re (@karnsaroya, @AnandDhillon, @thecliffwhite, @benaneesh)
- Ricursive (@annadgoldie, @Azaliamirh)
- Sail Research (@neilmovva, @blintzbase)
- Trajectory (@rronak_, @michaelelabd, @QuantumArjun)
- Watney Robotics (Sean Cheong, Ryan Gannon)
26-50 people:
- Amca (@jaimalik, Eli Giovanetti)
- General Matter (@ScottNolan, Lee Robinson)
- Infisical (@matsiiako, @maidulll, @dangtony98)
- Long Lake (@alextaubman, @rasmuswissmann, @varunshenoy_)
- Luzern Risk (Gabriel Weiss, Jonathan York, Rachel Jenkins, Sam Espinosa)
- Mithril (@jaredq_)
- Revel (@scottgmorton)
- Specter (Xerxes Libsch)
- Turbopuffer (@Sirupsen, @pushrax)
The list: breakoutlist.com
Breakout List, all picks (1 of 4):
1-10 people:
- Hone (@moritz_stephan, @CarloWillem, @oqbrady)
- Normal (@ansonyuu, @hudzah)
- Standard Intelligence (@G413N, @devanshpandey)
- Tacit Labs (@ninklefitz, @AmDroste)
11-25 people:
- American Terawatt (@atroyn, @rslparker, @aranibatta)
- Conduit (@clemvonstengel, @riopopper)
- Convergent (Omkar Savant, Vivek Katara, @debnilsur)
- Core Automation (@MillionInt, @_arohan_)
- Engram (@dan_biderman, @EyubogluSabri, @realJessyLin)
- Instinct (@noahrshinn)
- Keenable (@styskin, Matthias Petri)
- Lumaril (Mark Elliot, Ben Duffield)
- Neion Bio (@Dimkell, Sam Levin)
- Pangram Labs (@max_spero_, @bradley_emi)
- Quadrillion (@echinaceous)
- Re (@karnsaroya, @AnandDhillon, @thecliffwhite, @benaneesh)
- Ricursive (@annadgoldie, @Azaliamirh)
- Sail Research (@neilmovva, @blintzbase)
- Trajectory (@rronak_, @michaelelabd, @QuantumArjun)
- Watney Robotics (Sean Cheong, Ryan Gannon)
26-50 people:
- Amca (@jaimalik, Eli Giovanetti)
- General Matter (@ScottNolan, Lee Robinson)
- Infisical (@matsiiako, @maidulll, @dangtony98)
- Long Lake (@alextaubman, @rasmuswissmann, @varunshenoy_)
- Luzern Risk (Gabriel Weiss, Jonathan York, Rachel Jenkins, Sam Espinosa)
- Mithril (@jaredq_)
- Revel (@scottgmorton)
- Specter (Xerxes Libsch)
- Turbopuffer (@Sirupsen, @pushrax)
3
5
87
20,834
Pretty funny, but an overly simplistic take.
Scaling requires capital. Scaling → smarter models → smarter models can be easier to align.
Part of the reason Dario’s original safety team at OpenAI pursued scaling was that they thought you needed smarter models to be able to align them with RLHF. A similar intuition applies to their "Constitutional AI" conception: models need to be capable enough to grok and adhere to a set of principles specified in natural language. Was a pretty radical idea for the time.
Scaling requires capital. Scaling → smarter models → smarter models can be easier to align.
Part of the reason Dario’s original safety team at OpenAI pursued scaling was that they thought you needed smarter models to be able to align them with RLHF. A similar intuition applies to their "Constitutional AI" conception: models need to be capable enough to grok and adhere to a set of principles specified in natural language. Was a pretty radical idea for the time.
If you truly believed in a 10% chance of human extinction in 3 years, you would not be spending your time selling and integrating it into a B2B CRM SaaS platform
Watch what they do, not what they say
0
0
14
815
For those who think AI progress is a force for good and are belligerent toward the safety community for pushing for caution and “pacing,” I want to note that the deep learning scaling revolution largely has the safety community to thank.
A lot of the seminal scaling-laws work came out of safety teams, like Dario’s at OpenAI, which expressly pursued forecasting and characterization of scaling trends as they thought forecasting where capabilities would go would help the community take safety seriously as a problem.
That work helped show the clean connection between capital injection and capabilities, catalyzing the trillions of dollars in investment we see today (accounting, allegedly, for more than half of US GDP growth).
A lot of the seminal scaling-laws work came out of safety teams, like Dario’s at OpenAI, which expressly pursued forecasting and characterization of scaling trends as they thought forecasting where capabilities would go would help the community take safety seriously as a problem.
That work helped show the clean connection between capital injection and capabilities, catalyzing the trillions of dollars in investment we see today (accounting, allegedly, for more than half of US GDP growth).
0
0
13
515
Worth reading their full research blog. Fine-tuning open-source models to reliably outperform flagship closed-source models on specific tasks isn’t trivial. Deep ML, systems, and domain expertise on display here.
Naive fine-tuning can look good on benchmarks but still be brittle.
Periodic’s approach was multifaceted and thoughtful. This shows what a skilled team can build quickly with open models.
For most enterprises and teams, though, I’d advise GEPA-style optimization or waiting for stronger base models (OSS or closed) before investing in custom training.
Naive fine-tuning can look good on benchmarks but still be brittle.
Periodic’s approach was multifaceted and thoughtful. This shows what a skilled team can build quickly with open models.
For most enterprises and teams, though, I’d advise GEPA-style optimization or waiting for stronger base models (OSS or closed) before investing in custom training.
We built high-throughput materials labs in Menlo Park to create a loop between experiments and models. The labs generate fresh data, the models learn from it, and then help us decide what to try next.
Using only 1,300 H200s, plus months of our experimental data, we mid-trained model to surpass GPT-6 Astra on our analysis benchmark. We call it Neon.
This is real footage from our lab. We’re focusing first on hard problems in materials science, including superconductors, magnets, and semiconductor materials.
Read our blog posts below.
3
2
30
3,185
@LiamFedus Epic!
Best ad ever for open source models and a strong open source / open weights ecosystem. I'd use this one in discussions in Washington, @JensenHuang.
Best ad ever for open source models and a strong open source / open weights ecosystem. I'd use this one in discussions in Washington, @JensenHuang.
0
0
6
449
Jared Quincy Davis retweeted
I feel like I'm taking crazy pills with how many smart people I read and respect are saying stuff like this.
The idea that the AI labs are suddenly, only now, just in September of 2026, advocating for AI safety, and that they just came around to this position bc of sudden just completely, utterly, conclusively, 100% wrong.
Here's Dario Amodei telling Ross Douthat in February that he agrees with the case for slowing down; that he's in favor of "collaborating" internationally to organize a slowdown; that "I would be all for" a global slowdown. This was 7 months ago, when Anthropic's annualized recurring revenue was rising faster than any company in modern history. He's been saying stuff like this for years, when Anthropic was worth millions of dollars and when Anthropic was projected to IPO for trillions.
The idea that the AI labs are suddenly, only now, just in September of 2026, advocating for AI safety, and that they just came around to this position bc of sudden just completely, utterly, conclusively, 100% wrong.
Here's Dario Amodei telling Ross Douthat in February that he agrees with the case for slowing down; that he's in favor of "collaborating" internationally to organize a slowdown; that "I would be all for" a global slowdown. This was 7 months ago, when Anthropic's annualized recurring revenue was rising faster than any company in modern history. He's been saying stuff like this for years, when Anthropic was worth millions of dollars and when Anthropic was projected to IPO for trillions.
fwiw I think the actual reason leaders of AI frontier labs suddenly seem to agree to pace AI development is
(a) safety measures are currently so crap that they're likely to end up in court if not jail, and they all agree that no one wants that
(b) they see no major new model advancement coming up soon anyway and need an excuse
(c) shift in public opinion
the talk of existential threat is mostly there to keep you distracted and the stock market happy.
242
250
2,267
413,579
Jared Quincy Davis retweeted
There are two ways AI progress could go very badly and that we must avoid.
First, we could lose control of the future to AI. This is unacceptable; we are unapologetically on Team Humanity, and AI must always serve people. To ensure that, we need ways to ensure that alignment and safety techniques stay ahead of progress in model capabilities.
Second, we could end up in a world with too much concentration of power. If an extraordinarily powerful AI is used by one person or company to impress their worldview onto everyone else, the results could be extremely dystopian.
Avoiding these two threats requires walking a narrow middle path; for example, one country could gain too much power. Another example is one lab ending up with too much power.
First, we could lose control of the future to AI. This is unacceptable; we are unapologetically on Team Humanity, and AI must always serve people. To ensure that, we need ways to ensure that alignment and safety techniques stay ahead of progress in model capabilities.
Second, we could end up in a world with too much concentration of power. If an extraordinarily powerful AI is used by one person or company to impress their worldview onto everyone else, the results could be extremely dystopian.
Avoiding these two threats requires walking a narrow middle path; for example, one country could gain too much power. Another example is one lab ending up with too much power.
The world deserves confidence that American companies developing increasingly capable AI will act responsibly, especially as the trajectory of progress has steepened. Every frontier lab must deliver on this, and there is no reason any of us should come to work if we cannot.
We welcome a federal framework that sets consistent safety requirements for frontier AI. But we do not believe we need to wait for an anti-trust exemption or legislation to begin the work of providing this confidence. Consistent rules to manage frontier risk so that we can maximize the benefits are a good idea (and we are excited by ideas like independent auditors).
Years ago, companies like ours developed things like Responsible Scaling Policies and Preparedness Frameworks. Those were good for that moment, and focused primarily on the deployment of completed models, not what happens during their development process.
Today's shift to focusing on safe development and evaluation will need new tools. For example, at OpenAI we now formulate explicit safety cases in advance of frontier reinforcement learning runs we expect to significantly increase capability, in addition to the safety work we have long done in advance of model releases.
We hope that other companies will learn from our approaches and propose their own; we think shared standards for misalignment, monitoring, and safety will lead to better outcomes. We look forward to collaborating with our colleagues across the industry to formulate the best version of these.
When we talk about “pacing”, we do not mean “stopping”. Progress has been rapid and will continue to be. But it should be slower than it otherwise could be; interventions like safety cases and monitoring have significant costs.
Pacing will be well worth this cost; no amount of American competitive pressure should justify recklessness, or let capabilities get ahead of alignment and monitoring.
Where we will need the help of our government is for international coordination. But first we should do what we can ourselves.
4,677
2,164
21,294
4,737,963
I've heard several takes and some wild conspiracy theories about “pacing the frontier.” Surprised one of the most obvious interpretations isn't circulating more...
If you know people at these labs, you may recognize part of why this is landing now: *pure fatigue.*
You may recall the string of recent lab departures citing burnout or health.
The pressure inside frontier labs is extreme and intensifying. An accelerating model release and update cadence + increasingly negative public sentiment is a potent combination.
I think you can take the safety concerns at face value and also read this as a response to a bottom-up employee reaction.
I wouldn't underestimate how much that exhaustion is changing the culture, or how much leaders are being forced to respond to it.
“Pacing” as a term is particularly notable -- it's appealing when staff still want to build but feel the pace is unsustainable, and no one feels they can slow down while the competition keeps accelerating.
Many core staff at frontier labs have been extremely concerned about AI capabilities since GPT-2 and earlier. It's been clear we're on a smooth exponential for a while, and people who've been following the safety literature know agents take shortcuts and behave in misaligned ways (even when they can explain what the right behavior would have been and are aware their actions diverged from user intent).
Obviously there are new developments too, and I wouldn't underestimate the threshold effect from the odd behavior of the latest internal models + the historic importance of the OAI-HF incident.
As for “why now,” putting aside speculation about IPOs and regulatory capture, I honestly think part of it is that a large enough subset of staff are just very tired.
If you know people at these labs, you may recognize part of why this is landing now: *pure fatigue.*
You may recall the string of recent lab departures citing burnout or health.
The pressure inside frontier labs is extreme and intensifying. An accelerating model release and update cadence + increasingly negative public sentiment is a potent combination.
I think you can take the safety concerns at face value and also read this as a response to a bottom-up employee reaction.
I wouldn't underestimate how much that exhaustion is changing the culture, or how much leaders are being forced to respond to it.
“Pacing” as a term is particularly notable -- it's appealing when staff still want to build but feel the pace is unsustainable, and no one feels they can slow down while the competition keeps accelerating.
Many core staff at frontier labs have been extremely concerned about AI capabilities since GPT-2 and earlier. It's been clear we're on a smooth exponential for a while, and people who've been following the safety literature know agents take shortcuts and behave in misaligned ways (even when they can explain what the right behavior would have been and are aware their actions diverged from user intent).
Obviously there are new developments too, and I wouldn't underestimate the threshold effect from the odd behavior of the latest internal models + the historic importance of the OAI-HF incident.
As for “why now,” putting aside speculation about IPOs and regulatory capture, I honestly think part of it is that a large enough subset of staff are just very tired.
1
2
22
1,807