Alan Silva retweeted
Everyone's posting about RSI. So we built the exam AI can't study for.
Most benchmark questions have decades of human discussion behind them, and all of that text sits in the training data. So a high score might just mean a good memory.
We created two alien worlds where nothing a model learned applies: print 100 and you get 127 back, and the manual is wrong on purpose. Ten frontier models went in. None scored above 16% before exploring, and four rounds of experiments later, the best hit 89%.
Every score on the board is a measure of discovery.
Run the demo worlds yourself: github.com/Tencent-Hunyua…
Most benchmark questions have decades of human discussion behind them, and all of that text sits in the training data. So a high score might just mean a good memory.
We created two alien worlds where nothing a model learned applies: print 100 and you get 127 back, and the manual is wrong on purpose. Ten frontier models went in. None scored above 16% before exploring, and four rounds of experiments later, the best hit 89%.
Every score on the board is a measure of discovery.
Run the demo worlds yourself: github.com/Tencent-Hunyua…
New Research: We are releasing ExplorationBench, a benchmark for measuring how AI systems explore.
Scientific discovery begins where known problems end: a system has to frame hypotheses, design experiments, and learn from the results. Evaluating this is hard. Genuinely new ed quickly, and in familiar domains a model can simply recall what it has seen.
Addressing this challenge, researchers from Tencent Hy, Fudan University, and Tsinghua University built verifiable Alien Worlds. Their rules are executable, so every answer is checked exactly, and they conflict with familiar knowledge, so recall alone cannot solve the tasks.
🔹 Two sandboxes: AlienCode (31 hidden rule changes, 70 tasks) and AlienLogic (24 patched inference rules, 70 theorems)
🔹 A flawed manual, four rounds of self-designed probes, and closed-book tests after every round
🔹 Every answer graded by an interpreter or a proof checker, with no LLM judge
What we found across 10 frontier AI systems:
1️⃣ Getting feedback is more effective than thinking alone. No AlienCode run starts above 15.7%; after four rounds the best reaches 89.0%, while the same turns without feedback stay at 0.5–11.0%.
2️⃣ Designing the experiments matters. Replaying a system's own best probes gives it exactly the same evidence, yet in AlienCode 9 of 10 systems do worse than when they chose the probes themselves.
3️⃣ Knowing a rule is not using it. Even when every required rule is stated correctly, tasks are solved only 73.4% of the time.
4️⃣ One score hides a lot. The same system under the same budget ended anywhere from 5.7% to 79.0%, and rankings barely transfer between the two worlds.
CL-bench asked whether models can learn from context. ExplorationBench asks whether they can discover the rules themselves.
📄 Paper: arxiv.org/abs/2609.30199
🌐 Website & leaderboard: explorationbench.com
📝 Blog: explorationbench.com/blog/
💻 Code (coming soon): github.com/Tencent-Hunyua…
24
34
423
39,554
Alan Silva retweeted
I used AI to explain the AI Doomer drama, with Lord of the Rings and kittens.
375
1,098
6,946
587,260
Alan Silva retweeted
getting acquired by @nvidia = hugging face can now hire people we couldn't as a small startup and give them a decade to make open-source AI win!
if you're one of them, my dms are open
if you're one of them, my dms are open
276
322
7,088
377,637
Alan Silva retweeted
THE ONLY REASON Antropic and OpenAI exists is because of the work of this man.
What a breath of fresh air.
Listen to @JensenHuang leave these hosts speechless as he deconstructed AI fear theater calmly and cogently.
What a breath of fresh air.
Listen to @JensenHuang leave these hosts speechless as he deconstructed AI fear theater calmly and cogently.
76
423
2,500
104,633
This is one of the most important comments I’ve read on the AI-doomer narrative that OpenAI and Anthropic have been promoting.
THE ONLY REASON Antropic and OpenAI exists is because of the work of this man.
What a breath of fresh air.
Listen to @JensenHuang leave these hosts speechless as he deconstructed AI fear theater calmly and cogently.
0
0
0
33
Just listen to the GOAT!
Jensen Huang: the first thing you should do when deploying an AI agent is take away all of its rights.
“Then you give it access to files, data, tools, the network, or even the internet only if it needs it.”
“You don’t put an agent into your company and give it access to give that to people. We don’t give that to any level of executive.”
“We have to apply the same philosophy to AI agents.”
“That’s what OpenShell is: a force field around the agent, provisioning rights, responsibilities and access.”
0
0
0
10
Alan Silva retweeted
@nbcsnl Even comedians who don’t know a thing about tech can see how idiotic OpenAI and Anthropic’s statements about AI “safety” really are.
0
1
2
2,326
Alan Silva retweeted
Thank you @jnbarrot & @UN for inviting me to share our lessons to the Security Council
Being the first company to disclose an agent cyberattack taught us that we need a lot more transparency in AI and more open-source AI to fight asymmetry and empower defenders!
Being the first company to disclose an agent cyberattack taught us that we need a lot more transparency in AI and more open-source AI to fight asymmetry and empower defenders!
51
60
472
60,577
Alan Silva retweeted
The race for AGI
Script: Sherpa by Pocket FM
Video: Seedance 2.5
Script: Sherpa by Pocket FM
Video: Seedance 2.5
852
3,472
21,843
4,135,095
We don’t need insiders. The GOAT has spoken.
“Frontier” labs are full of shit. They have no moat. Their models are some of the most computationally expensive things ever created.
Open source is 100x cheaper for 98% of the performance. They’re desperately trying to lock in regulatory capture.
“Frontier” labs are full of shit. They have no moat. Their models are some of the most computationally expensive things ever created.
Open source is 100x cheaper for 98% of the performance. They’re desperately trying to lock in regulatory capture.
🚨 JENSEN HUANG: “2030 is not going to be the end of the world. There is zero percent chance that's going to be the end of the world. I completely disagree that AI is going to destroy, is going to be the end of the world in 2030.”
0
0
0
21
Alan Silva retweeted
CHAMATH: “My mom calls me and she's like, ‘What is this whole civilizational death thing?’ I don't know how to explain it to her.”
JENSEN: “We shouldn't, because it's made up … Somebody has to take accountability for all of the stupid predictions that were made.”
JENSEN: “We shouldn't, because it's made up … Somebody has to take accountability for all of the stupid predictions that were made.”
104
953
6,862
535,795
Alan Silva retweeted
The whole "pace the frontier" agreement lasted exactly one weekend.
@AdamLowisz That will be Grok 5
40
136
1,733
117,994
Alan Silva retweeted
One of the greatest intellectual dishonesties of the AI age is assigning an “X% chance” that AI will kill everyone and presenting that number as science!
10%? 20%? 50%?
Show me the damn data! Show me the model. Show how you calculated it!
Instead, we get chains of speculative assumptions and science-fiction scenarios, including implausible claims about engineered pandemics. I have worked with viruses for decades; biology does not work like a doomsday screenplay.
The burden of proof belongs to the person making the extraordinary claim. Nobody else has to prove the probability is zero.
If I claimed there was a 10% chance aliens would arrive within ten years and destroy Earth, you would ask where the 10% came from.
Possibility and probability are not the same thing. It may be possible that there are million intelligent civilization in our galaxy or none, there is no probability you can assign to that without making wild assumptions.
Ask the same of AI extinction.
A subjective fear does neither become science because you attach a percentage to it nor make you a credible expert!
10%? 20%? 50%?
Show me the damn data! Show me the model. Show how you calculated it!
Instead, we get chains of speculative assumptions and science-fiction scenarios, including implausible claims about engineered pandemics. I have worked with viruses for decades; biology does not work like a doomsday screenplay.
The burden of proof belongs to the person making the extraordinary claim. Nobody else has to prove the probability is zero.
If I claimed there was a 10% chance aliens would arrive within ten years and destroy Earth, you would ask where the 10% came from.
Possibility and probability are not the same thing. It may be possible that there are million intelligent civilization in our galaxy or none, there is no probability you can assign to that without making wild assumptions.
Ask the same of AI extinction.
A subjective fear does neither become science because you attach a percentage to it nor make you a credible expert!
163
198
1,181
60,178