Pinned Tweet
My predictions for 2026:
Coding and Mathematics AGI
- METR 50% time horizons above 24 hours - my mean estimate is 30.8 hours, 2 day time horizons possible within frontier labs when accounting for 60 day lag
- if 2025 was the year of agents, then 2026 will be the year of multi-agent systems
- agents delegating work to subagents -> the start of the agent economy and the great unhobbling!
Most of our current math and coding benchmarks will get saturated!
- Epoch Capabilities Index ( > 175 )
- FrontierMath Levels 1-3 ( > 95% )
- ARC-AGI 1 and 2 ( > 95% )
- SimpleQA verified ( > 95% )
- Simple-Bench ( > 90% )
- SWE-Bench-verified ( > 90% )
- Terminal-Bench 2 ( > 90% )
- WeirdML v2 ( > 85% )
- Humanities Last Exam ( > 80% )
- FrontierMath Level 4 ( > 75% )
- Cybench ( > 70% )
- GDPval ( > 70 % win rate, no ties)
- GSO ( > 65% )
- ARC-AGI-3 ( > 60% and > 80% if they go for o3-preview comparable compute budgets or continual learning breakthrough happens)
- more evals like gdpval that capture economic value of models and systems
- big focus white collar work and large acceleration of science: specifically i see acceleration in medicine, biology, chemistry, finance, legal, administrative work
- automation of white collar work will be enabled by having reliable and fast computer use agents
- reliable computer use agents will also have implications for how you use the internet. this is OpenAI's big goal: become the hub to the internet and delegate shopping and whatever to agents!
Big models launches to get hyped for in 2026:
- Claude 5 - Claude 5.5
- Gemini 3.5 - Gemini-4
- GPT-5.3 - GPT-6
(everything in between possible, but Gemini 4 ~ 80%, Claude 5.5 ~ 70%, GPT-6 ~ 60% likely before 2027)
- DeepSeek-V4
- Grok-5
- Qwen-4
- Kimi-K3, GLM-5, MiniMax M3
- more korean models and a bunch of american open-source models :)
The gap between closed and open labs will narrow in H1 2026 due to DeepSeek-V4, then widen in the later half of the year, especially on economically valuable tasks.
Closed models will be much more reliable. But we will still have Opus 4.5+ level open models by the end of 2026.
Most frontier models will be around 5-10T params. If we see GPT-6 and Gemini-4 at the end of 2026 10T+ param models are possible. These models + harnesses will be the first not research agents. We should also see much better live models with voice and video mode.
Model architecture:
- we will see both, more efficient architectures and more expressive architectures!
- hybrid architectures for even longer context windows, diffusion models for speed on edge devices, but also models that double down on full attention or even more expressive attention mechanisms
- looped language models, other recurrent architectures and continual learning will enable much smaller reasoning models! (TRM on ARC-AGI has paved the way for the reasoning core)
- big improvements in reasoning efficiency
in my 2025 prediction I included a prediction for 2026 that I stand by:
- "someone (Anthropic) figures out efficient test-time-training [...], this will be the next paradigm for 2026 and lead to superintelligence"
General outlook and some random thoughts:
- it will be clear to everybody that Anthropic has the mandate and is ahead of everyone else
- OpenAI, Anthropic and Google will remain frontier labs
- decent chance that Anthropic overtakes OpenAI's valuation and both are valued > 1T
- DeepSeek will join them with V4 as THE chinese frontier lab
- xAI will likely repeat Grok-4, Grok-5 will be great on benchmarks but Elon persists on slop-maxxing the model
- AI generated video content will take off with Veo-4 and Sora-3, consistent minute long videos will be possible
- embodied intelligence will start to take off by RL through world models
- full self-driving solved, waymo and tesla everywhere
- the stock market will have a 20%+ drawdown
- 15% chance of OpenAI going bankrupt and getting acquired by Microsoft due to collapse of oracle or a market crash, caused by rapidly deteriorating economic situation (unemployment, inflation)
- push against AI will become a common theme in most advanced western economies as unemployment rises
- populist right wing parties continue to gain traction in europe
- trump/republicans will lose midterm elections
Coding and Mathematics AGI
- METR 50% time horizons above 24 hours - my mean estimate is 30.8 hours, 2 day time horizons possible within frontier labs when accounting for 60 day lag
- if 2025 was the year of agents, then 2026 will be the year of multi-agent systems
- agents delegating work to subagents -> the start of the agent economy and the great unhobbling!
Most of our current math and coding benchmarks will get saturated!
- Epoch Capabilities Index ( > 175 )
- FrontierMath Levels 1-3 ( > 95% )
- ARC-AGI 1 and 2 ( > 95% )
- SimpleQA verified ( > 95% )
- Simple-Bench ( > 90% )
- SWE-Bench-verified ( > 90% )
- Terminal-Bench 2 ( > 90% )
- WeirdML v2 ( > 85% )
- Humanities Last Exam ( > 80% )
- FrontierMath Level 4 ( > 75% )
- Cybench ( > 70% )
- GDPval ( > 70 % win rate, no ties)
- GSO ( > 65% )
- ARC-AGI-3 ( > 60% and > 80% if they go for o3-preview comparable compute budgets or continual learning breakthrough happens)
- more evals like gdpval that capture economic value of models and systems
- big focus white collar work and large acceleration of science: specifically i see acceleration in medicine, biology, chemistry, finance, legal, administrative work
- automation of white collar work will be enabled by having reliable and fast computer use agents
- reliable computer use agents will also have implications for how you use the internet. this is OpenAI's big goal: become the hub to the internet and delegate shopping and whatever to agents!
Big models launches to get hyped for in 2026:
- Claude 5 - Claude 5.5
- Gemini 3.5 - Gemini-4
- GPT-5.3 - GPT-6
(everything in between possible, but Gemini 4 ~ 80%, Claude 5.5 ~ 70%, GPT-6 ~ 60% likely before 2027)
- DeepSeek-V4
- Grok-5
- Qwen-4
- Kimi-K3, GLM-5, MiniMax M3
- more korean models and a bunch of american open-source models :)
The gap between closed and open labs will narrow in H1 2026 due to DeepSeek-V4, then widen in the later half of the year, especially on economically valuable tasks.
Closed models will be much more reliable. But we will still have Opus 4.5+ level open models by the end of 2026.
Most frontier models will be around 5-10T params. If we see GPT-6 and Gemini-4 at the end of 2026 10T+ param models are possible. These models + harnesses will be the first not research agents. We should also see much better live models with voice and video mode.
Model architecture:
- we will see both, more efficient architectures and more expressive architectures!
- hybrid architectures for even longer context windows, diffusion models for speed on edge devices, but also models that double down on full attention or even more expressive attention mechanisms
- looped language models, other recurrent architectures and continual learning will enable much smaller reasoning models! (TRM on ARC-AGI has paved the way for the reasoning core)
- big improvements in reasoning efficiency
in my 2025 prediction I included a prediction for 2026 that I stand by:
- "someone (Anthropic) figures out efficient test-time-training [...], this will be the next paradigm for 2026 and lead to superintelligence"
General outlook and some random thoughts:
- it will be clear to everybody that Anthropic has the mandate and is ahead of everyone else
- OpenAI, Anthropic and Google will remain frontier labs
- decent chance that Anthropic overtakes OpenAI's valuation and both are valued > 1T
- DeepSeek will join them with V4 as THE chinese frontier lab
- xAI will likely repeat Grok-4, Grok-5 will be great on benchmarks but Elon persists on slop-maxxing the model
- AI generated video content will take off with Veo-4 and Sora-3, consistent minute long videos will be possible
- embodied intelligence will start to take off by RL through world models
- full self-driving solved, waymo and tesla everywhere
- the stock market will have a 20%+ drawdown
- 15% chance of OpenAI going bankrupt and getting acquired by Microsoft due to collapse of oracle or a market crash, caused by rapidly deteriorating economic situation (unemployment, inflation)
- push against AI will become a common theme in most advanced western economies as unemployment rises
- populist right wing parties continue to gain traction in europe
- trump/republicans will lose midterm elections
99
106
1,108
707,125
>"yeah I'm sure we have reached the top of the AI bubble"
> Anthropic: Introducing Opus 5.5
> oh
> Anthropic: Introducing Opus 5.5
> oh
30
13
377
23,173
Lisan al Gaib retweeted
Tinker's new StableReinforce objective is Reinforce + Score Centering. Here's how I reverse-engineered it 🧵
Off-policy RL is difficult; runs can suddenly collapse because of async RL staleness or numerical mismatch.
StableReinforce is a new method on Tinker that keeps your RL runs stable, even when learning drifts from sampling.
tinker-docs.thinkingmachines.ai/tinker/losses/…
14
52
558
46,435
a lot of people find Kimi-K3.1 disappointing
Kimi 3.1 is the best released Chinese model but progression over 3.0 in 3 months is underwhelming.
It’s Opus5.0/GPT-6 Sol level rather than Opus5.5/6.1 level. Not bad, but DeepSeek has clearly progressed faster with its RL than Moonshot. In cyber & some SWE tasks, 1017 is actuDS flash update will catch up to this on SWE & work tasks.
So then what do you do if you are Moonshot? Well, I wonder if it’s just pouring most of its energy on the larger K4. Among top Chinese labs, it seems best at pre training & worst @ post training.
So it’s best bet is keep scaling up. Its models are not efficient like DeepSeek models. It has never pretended otherwise. Even at current px level, it’s overwhelmed by demand so why should it charge less? You should be using DeepSeek if you are just doing regular work stuff.
28
5
208
11,693
On another derived task where the knowledge recall and graph construction components are removed from LisanBench OpenAI models are also extremely strong
Models are given 100 words as a graph and have to find the longest simple path given a starting word. This is closest to
Models are given 100 words as a graph and have to find the longest simple path given a starting word. This is closest to
it's good to know that OpenAI models are at least SOTA on a subtask of LisanBench:
"When scoring the models without the edit-distance one rule (all the other rules still active) the ranking changes completely:
1. GPT 6 Astra medium: 56,140 (+659%)
2. Gemini 4 Argon high: 51,312 (+579%)
3. GPT 6.1 Sol medium: 49,887 (+731%)"
ALL THREE CONFIRMED LOOPERS AT THE TOP 👀
9
1
32
4,818
it's good to know that OpenAI models are at least SOTA on a subtask of LisanBench:
"When scoring the models without the edit-distance one rule (all the other rules still active) the ranking changes completely:
1. GPT 6 Astra medium: 56,140 (+659%)
2. Gemini 4 Argon high: 51,312 (+579%)
3. GPT 6.1 Sol medium: 49,887 (+731%)"
ALL THREE CONFIRMED LOOPERS AT THE TOP 👀
"When scoring the models without the edit-distance one rule (all the other rules still active) the ranking changes completely:
1. GPT 6 Astra medium: 56,140 (+659%)
2. Gemini 4 Argon high: 51,312 (+579%)
3. GPT 6.1 Sol medium: 49,887 (+731%)"
ALL THREE CONFIRMED LOOPERS AT THE TOP 👀
Ad - Meta provided free access to Muse Wonder models. The tests and conclusions are my own.
This is a massive update for LisanBench:
- I finally bit the bullet and tested multiple reasoning levels. We now have some sweet sweet data.
- Muse Wonder Full and Muse Wonder perform Bench and have crazy scaling and very even spacing between reasoning efforts. Just based on scores both Muse models with max reasoning effort rank #1 and #2. However, they are still slightly less efficient than Anthropic's Opus 5.5
- DeepSeek-V4.1-Flash 1017 is also very impressive and on the perf-cost frontier together with Haiku 5.5.
- Notably Gemini 4 Argon low and medium, as well as Sonnet 5.5 low and medium produce no reasoning tokens and respond instantly, but still score high.
Gemini 4 Argon low takes the #1 spot on the non-thinking leaderboard (it's a dirty looper).
- Interestingly GPT-6.1-Sol, and GPT-6 models perform poorly as they are not good at following the edit-distance one rule. Argon high shows similar behavior.
For GPT models I think the reason could be related to their very terse reasoning style. But since I haven't seen Gemini 4 Argon's raw reasoning I'm not super confident in that theory. Another explanation could be that it's related to dirty loopers.
- When scoring the models without the edit-distance one rule (all the other rules still active) the ranking changes completely:
1. GPT 6 Astra medium: 56,140 (+659%)
2. Gemini 4 Argon high: 51,312 (+579%)
3. GPT 6.1 Sol medium: 49,887 (+731%)
- The worst offenders in terms of cheating are Gemini 4 Argon and DeepSeek V4.1 Flash 1017.
Gemini exposes an entirely new bug in the scorer using non-ASCII characters and DeepSeek has multiple longer chains with up to 64 consecutive two-letter words. When I deliberately RL-hillclimbed LisanBench I saw the same behavior of collapsing word lengths.
Anthropic models also seem very optimized for this game, but in less obvious ways. They keep reusing the same long chains across multiple different starting words.
- The minimum cost of reaching 2000, 5000 and 10000 LisanBench points has declined by 22-25% per month or halving every 2.4-2.8 months
8
2
85
11,873
Lisan al Gaib retweeted
Ad - Meta provided free access to Muse Wonder models. The tests and conclusions are my own.
This is a massive update for LisanBench:
- I finally bit the bullet and tested multiple reasoning levels. We now have some sweet sweet data.
- Muse Wonder Full and Muse Wonder perform Bench and have crazy scaling and very even spacing between reasoning efforts. Just based on scores both Muse models with max reasoning effort rank #1 and #2. However, they are still slightly less efficient than Anthropic's Opus 5.5
- DeepSeek-V4.1-Flash 1017 is also very impressive and on the perf-cost frontier together with Haiku 5.5.
- Notably Gemini 4 Argon low and medium, as well as Sonnet 5.5 low and medium produce no reasoning tokens and respond instantly, but still score high.
Gemini 4 Argon low takes the #1 spot on the non-thinking leaderboard (it's a dirty looper).
- Interestingly GPT-6.1-Sol, and GPT-6 models perform poorly as they are not good at following the edit-distance one rule. Argon high shows similar behavior.
For GPT models I think the reason could be related to their very terse reasoning style. But since I haven't seen Gemini 4 Argon's raw reasoning I'm not super confident in that theory. Another explanation could be that it's related to dirty loopers.
- When scoring the models without the edit-distance one rule (all the other rules still active) the ranking changes completely:
1. GPT 6 Astra medium: 56,140 (+659%)
2. Gemini 4 Argon high: 51,312 (+579%)
3. GPT 6.1 Sol medium: 49,887 (+731%)
- The worst offenders in terms of cheating are Gemini 4 Argon and DeepSeek V4.1 Flash 1017.
Gemini exposes an entirely new bug in the scorer using non-ASCII characters and DeepSeek has multiple longer chains with up to 64 consecutive two-letter words. When I deliberately RL-hillclimbed LisanBench I saw the same behavior of collapsing word lengths.
Anthropic models also seem very optimized for this game, but in less obvious ways. They keep reusing the same long chains across multiple different starting words.
- The minimum cost of reaching 2000, 5000 and 10000 LisanBench points has declined by 22-25% per month or halving every 2.4-2.8 months
This is a massive update for LisanBench:
- I finally bit the bullet and tested multiple reasoning levels. We now have some sweet sweet data.
- Muse Wonder Full and Muse Wonder perform Bench and have crazy scaling and very even spacing between reasoning efforts. Just based on scores both Muse models with max reasoning effort rank #1 and #2. However, they are still slightly less efficient than Anthropic's Opus 5.5
- DeepSeek-V4.1-Flash 1017 is also very impressive and on the perf-cost frontier together with Haiku 5.5.
- Notably Gemini 4 Argon low and medium, as well as Sonnet 5.5 low and medium produce no reasoning tokens and respond instantly, but still score high.
Gemini 4 Argon low takes the #1 spot on the non-thinking leaderboard (it's a dirty looper).
- Interestingly GPT-6.1-Sol, and GPT-6 models perform poorly as they are not good at following the edit-distance one rule. Argon high shows similar behavior.
For GPT models I think the reason could be related to their very terse reasoning style. But since I haven't seen Gemini 4 Argon's raw reasoning I'm not super confident in that theory. Another explanation could be that it's related to dirty loopers.
- When scoring the models without the edit-distance one rule (all the other rules still active) the ranking changes completely:
1. GPT 6 Astra medium: 56,140 (+659%)
2. Gemini 4 Argon high: 51,312 (+579%)
3. GPT 6.1 Sol medium: 49,887 (+731%)
- The worst offenders in terms of cheating are Gemini 4 Argon and DeepSeek V4.1 Flash 1017.
Gemini exposes an entirely new bug in the scorer using non-ASCII characters and DeepSeek has multiple longer chains with up to 64 consecutive two-letter words. When I deliberately RL-hillclimbed LisanBench I saw the same behavior of collapsing word lengths.
Anthropic models also seem very optimized for this game, but in less obvious ways. They keep reusing the same long chains across multiple different starting words.
- The minimum cost of reaching 2000, 5000 and 10000 LisanBench points has declined by 22-25% per month or halving every 2.4-2.8 months
30
8
169
74,621
Ad - Meta provided free access to Muse Wonder models. The tests and conclusions are my own.
This is a massive update for LisanBench:
- I finally bit the bullet and tested multiple reasoning levels. We now have some sweet sweet data.
- Muse Wonder Full and Muse Wonder perform Bench and have crazy scaling and very even spacing between reasoning efforts. Just based on scores both Muse models with max reasoning effort rank #1 and #2. However, they are still slightly less efficient than Anthropic's Opus 5.5
- DeepSeek-V4.1-Flash 1017 is also very impressive and on the perf-cost frontier together with Haiku 5.5.
- Notably Gemini 4 Argon low and medium, as well as Sonnet 5.5 low and medium produce no reasoning tokens and respond instantly, but still score high.
Gemini 4 Argon low takes the #1 spot on the non-thinking leaderboard (it's a dirty looper).
- Interestingly GPT-6.1-Sol, and GPT-6 models perform poorly as they are not good at following the edit-distance one rule. Argon high shows similar behavior.
For GPT models I think the reason could be related to their very terse reasoning style. But since I haven't seen Gemini 4 Argon's raw reasoning I'm not super confident in that theory. Another explanation could be that it's related to dirty loopers.
- When scoring the models without the edit-distance one rule (all the other rules still active) the ranking changes completely:
1. GPT 6 Astra medium: 56,140 (+659%)
2. Gemini 4 Argon high: 51,312 (+579%)
3. GPT 6.1 Sol medium: 49,887 (+731%)
- The worst offenders in terms of cheating are Gemini 4 Argon and DeepSeek V4.1 Flash 1017.
Gemini exposes an entirely new bug in the scorer using non-ASCII characters and DeepSeek has multiple longer chains with up to 64 consecutive two-letter words. When I deliberately RL-hillclimbed LisanBench I saw the same behavior of collapsing word lengths.
Anthropic models also seem very optimized for this game, but in less obvious ways. They keep reusing the same long chains across multiple different starting words.
- The minimum cost of reaching 2000, 5000 and 10000 LisanBench points has declined by 22-25% per month or halving every 2.4-2.8 months
This is a massive update for LisanBench:
- I finally bit the bullet and tested multiple reasoning levels. We now have some sweet sweet data.
- Muse Wonder Full and Muse Wonder perform Bench and have crazy scaling and very even spacing between reasoning efforts. Just based on scores both Muse models with max reasoning effort rank #1 and #2. However, they are still slightly less efficient than Anthropic's Opus 5.5
- DeepSeek-V4.1-Flash 1017 is also very impressive and on the perf-cost frontier together with Haiku 5.5.
- Notably Gemini 4 Argon low and medium, as well as Sonnet 5.5 low and medium produce no reasoning tokens and respond instantly, but still score high.
Gemini 4 Argon low takes the #1 spot on the non-thinking leaderboard (it's a dirty looper).
- Interestingly GPT-6.1-Sol, and GPT-6 models perform poorly as they are not good at following the edit-distance one rule. Argon high shows similar behavior.
For GPT models I think the reason could be related to their very terse reasoning style. But since I haven't seen Gemini 4 Argon's raw reasoning I'm not super confident in that theory. Another explanation could be that it's related to dirty loopers.
- When scoring the models without the edit-distance one rule (all the other rules still active) the ranking changes completely:
1. GPT 6 Astra medium: 56,140 (+659%)
2. Gemini 4 Argon high: 51,312 (+579%)
3. GPT 6.1 Sol medium: 49,887 (+731%)
- The worst offenders in terms of cheating are Gemini 4 Argon and DeepSeek V4.1 Flash 1017.
Gemini exposes an entirely new bug in the scorer using non-ASCII characters and DeepSeek has multiple longer chains with up to 64 consecutive two-letter words. When I deliberately RL-hillclimbed LisanBench I saw the same behavior of collapsing word lengths.
Anthropic models also seem very optimized for this game, but in less obvious ways. They keep reusing the same long chains across multiple different starting words.
- The minimum cost of reaching 2000, 5000 and 10000 LisanBench points has declined by 22-25% per month or halving every 2.4-2.8 months
30
8
169
74,621
Anthropic and Muse Wonder models are dominating in terms of tokens/step or tokens/point
0
0
14
1,839
"do not put on the paid partnership label"
this sounds like something you would actually go to prison for in europe
this sounds like something you would actually go to prison for in europe
That's how Kimi AI are promoting their new model Kimi K3.1 pro and I know all the influencers working with them and none of them are honest,I have posted about it myself but I didn't join this and never used their templates that they provide.
PROPAGANDA, I have more to share na have follow up for this
21
11
284
9,188
Lisan al Gaib retweeted
Last month a large team of agents used over 100 billion tokens to resolve the Navier-Stokes Millennium Prize problem. Now we're using a similar approach for red-teaming to find and fix infra vulnerabilities.
To make sure our frontier models don't cause another HuggingFace incident, we built an offline world that they use during training. It provides everything that the models need in a fully cached, indexed, or synthetic manner.
We then built a huge team of our most cyber-capable models and told them to collaborate to find zero-days in this containment infrastructure.
While they didn't find a path to the internet that wouldn't be caught by our CoT monitors, they did find ways to escape popular VMs, exploit our network infrastructure, and even ways to communicate with other agents running in separate VMs by using hardware side channels.
We used this to dramatically harden our infra, and we also upstreamed a ton of patches to open-sources packages.
86
121
1,736
64,417
won't this make agents even more eval aware and shizo?
how do you know that the agents will behave the same on the real, live internet?
how do you know that the agents will behave the same on the real, live internet?
11
2
121
7,440
