Nikil Ravi retweeted
I’m so glad Elie is going on this campaign. The rest of the world doesn’t know what AIME vs GeneBench Pro is so there’s going to be people getting swindled if we don’t try and communicate how much of a scam things like this are
no no no tell me this is not real, "the smartest sovereign AI" but you use benchmark from 2 years ago and without ONE agentic benchmark ??? "intelligence you control" but you can't benchmark yourself and it's not open ???
2
1
59
3,869
Nikil Ravi retweeted
Just because you have a cake, doesn't mean you can attempt to enter someone's office unannounced
10
1
33
3,739
Nikil Ravi retweeted
We need to move away from what i call “launch slop”
Fast cuts, black and white, historical film, rocket ships. Clips of some
philosopher talking.
It was cool when founders fund did it. Now it looks like shit. Every YC company has the same video. We need a new aesthetic.
Fast cuts, black and white, historical film, rocket ships. Clips of some
philosopher talking.
It was cool when founders fund did it. Now it looks like shit. Every YC company has the same video. We need a new aesthetic.
73
18
608
29,634
Yep, this is the fundamental tradeoff you make. In theory you can come up with fun ideas to get past this (eg. have a select advisory council that audits private benchmarks but is not connected to model developers) but really is not practically feasible simply because it would be too slow.
Private benchmarks often show open models are further behind, but the problem with private benchmarks is you don’t have the community to squash the inevitable bugs for you. I'm 50/50 on if I think private or open-source is the best approach.
0
0
5
320
Nikil Ravi retweeted
"'Evaluator' as a role has the potential to be as prevalent as 'software developer'"
Having 6.5k stars on @harborframework repo is crazy
The group of people doing evals is still so small compared to developers as a whole
But we know literally every company who uses AI will be running their own private evals at some point...
"Evaluator" as a role has the alent as "software developer"
0
2
22
1,687
Nikil Ravi retweeted
Dario reveals the people who called his GPT-2 fears "pseudoscience" in 2019 are now all applying for jobs at AI labs
"I personally was super freaked out by [GPT-2]. I looked at it and I said, my God, you just scale this thing up another factor of a million and it'll be better ng. This is the defining economic and scientific moment of the modern age. We just don't know it yet. This is going to determine the fate of nations and economies and the future of humanity."
"Maybe 10% of the people around me had exactly the same reaction, and it was often the smartest ones. 90% of the people around me, because GPT-2's capabilities in any particular area were not that impressive, it could translate languages, summarize a document and write a few lines of code here and there. It was the broadness and the generality."
"But people would look at it and say, in any particular area, that doesn't seem so smart. And so many people just shrugged and they just dismissed it."
"I just look at it and I say it's the same thing over and over again. The limitations of the models, six months from now, the things that seemed hard, they're going to be easy. 12 months from now, the things that seemed impossible, the models are going to be doing them regularly."
"It's everything we saw with GPT-2; we're seeing it over and over again in wider and wider swaths of the world. And the world is gradually finally being convinced."
Daniela Amodei: "I remember at the time there were some very well-respected computer scientists and AI and ML leaders who basically bullied some of the researchers who worked on [GPT-2] on Twitter and were like, this is stupid. We don't know why you guys are even talking, this model is so incapable."
"It was like you were sort of a quack if you believed this was a really significant moment."
Dario: "That's all been memory-holed. All the researchers who were like, oh, this is pseudoscience, it doesn't make sense, all of them are applying for jobs at the LLM companies."
"We never say it out loud to any particular one, but we look at them and they're like, we remember what you were saying in 2019. You have no credibility on this matter."
"I personally was super freaked out by [GPT-2]. I looked at it and I said, my God, you just scale this thing up another factor of a million and it'll be better ng. This is the defining economic and scientific moment of the modern age. We just don't know it yet. This is going to determine the fate of nations and economies and the future of humanity."
"Maybe 10% of the people around me had exactly the same reaction, and it was often the smartest ones. 90% of the people around me, because GPT-2's capabilities in any particular area were not that impressive, it could translate languages, summarize a document and write a few lines of code here and there. It was the broadness and the generality."
"But people would look at it and say, in any particular area, that doesn't seem so smart. And so many people just shrugged and they just dismissed it."
"I just look at it and I say it's the same thing over and over again. The limitations of the models, six months from now, the things that seemed hard, they're going to be easy. 12 months from now, the things that seemed impossible, the models are going to be doing them regularly."
"It's everything we saw with GPT-2; we're seeing it over and over again in wider and wider swaths of the world. And the world is gradually finally being convinced."
Daniela Amodei: "I remember at the time there were some very well-respected computer scientists and AI and ML leaders who basically bullied some of the researchers who worked on [GPT-2] on Twitter and were like, this is stupid. We don't know why you guys are even talking, this model is so incapable."
"It was like you were sort of a quack if you believed this was a really significant moment."
Dario: "That's all been memory-holed. All the researchers who were like, oh, this is pseudoscience, it doesn't make sense, all of them are applying for jobs at the LLM companies."
"We never say it out loud to any particular one, but we look at them and they're like, we remember what you were saying in 2019. You have no credibility on this matter."
54
96
2,059
347,068
Nikil Ravi retweeted
The best defense is offense: if you can't stress test your own systems, somebody else will.
I built 𝚌𝚙𝚞-𝚌𝚑𝚊𝚝, a reproduction of @OpenAI's findings of covert communication using the CPU.
It runs on a generic host with 2+ Firecracker microVMs with no internet access. Works on x86 and even my M5 chip.
I was able to chat via 3 separate mechanisms:
① CPU tion timing
③ DRAM bandwidth contention
… across multiple CPUs, defeating all Firecracker's production recommendations (no SMT, no KSM, tenants pinned to different cores.)
Essentially, if shared hardware is involved, there's a good chance your agents can communicate covertly. And with really decent latency.
The good news: this is decidedly different from a *side-channel attack* where the agents could steal information (Meltdown/Spectre), a category of attack that providers like Vercel Sandbox mitigate. Further, in our real-world environment, these 'whispering' techniques are much harder to pull off and easier for us to monitor and detect (sampling hardware counters + lots of at-scale statistical data.)
Noteworthy: ironically, OpenAI and Anthropic's models refused to do this work. Open weight models got the job done. I'm worried the labs' stance might backfire and get in the way of helping developers build correct and secure systems. e.g: building a secure sandbox requires having this kind of analysis, and without it, we'll have a less secure world.
3
1
32
4,617
Nikil Ravi retweeted
The HKS TAIGR lab website is now up! Building a forward-looking, AI gov research group at the HKS is an ongoing process, and I'm excited about the next stages. @AnkaReuel
Speaking of which -- PhD and Master's apps are due in early Dec.
belfercenter.org/technical-ai-g…
Speaking of which -- PhD and Master's apps are due in early Dec.
belfercenter.org/technical-ai-g…
0
10
82
3,190
Was very cool to see the eval community together. Thanks @typewriters @arcprize @SnorkelAI for organizing this!
We co-hosted a dinner in SF with @SnorkelAI and our favorite people building and leveraging evals
After years of hearing about how creating AI standards will lead to a race to the top, I've been confounded that experts on the ground developing, deploying and maintaining ivate sector + nonprofits) were not talking to one another or collaborating on frameworks to help scale the evaluator ecosystem
Without shared knowledge around how to create these measurement tools, we'll never know if AI systems are meeting the standards
The dinner came after @SnorkelAI announced their $30M open benchmarking grant program and before the first @arcprize Research Summit in Boston
This area of measurement science is new and we'll only get to our desired end state by working together (example on the state of the world re: third party AI auditing - some folks on the nonprofit side had not heard of Snorkel, which just raised a $350M Series E and their business requires shipping quality benchmarks across domains). There's a lot of we can learn from one another
Thanks to @ValsAI , @EpochAIResearch , @METR_Evals , @Harvard , @GoogleDeepMind, @midjourney, @nvidia, @IFP, @NIST, @a16z, @LaudeInstitute, @GriffinCMU, @OpenAI, @SemiAnalysis_ , @Meta, @MLCommons, @MTSlive and @alice_dot_io for attending, and @arcprize for lifting all boats
0
3
11
698
Nikil Ravi retweeted
One of the best-curated events I’ve attended this year! Everyone I met had thoughtful, sensible perspectives on where AI evaluations and the ecosystem need to go. Left with new friends, collaborations, and even a potential funder for my new lab that’s gonna be working on evals!
Big thanks to @typewriters @SnorkelAI @arcprize for bringing such a great crowd together 🫶
Big thanks to @typewriters @SnorkelAI @arcprize for bringing such a great crowd together 🫶
We co-hosted a dinner in SF with @SnorkelAI and our favorite people building and leveraging evals
After years of hearing about how creating AI standards will lead to a race to the top, I've been confounded that experts on the ground developing, deploying and maintaining ivate sector + nonprofits) were not talking to one another or collaborating on frameworks to help scale the evaluator ecosystem
Without shared knowledge around how to create these measurement tools, we'll never know if AI systems are meeting the standards
The dinner came after @SnorkelAI announced their $30M open benchmarking grant program and before the first @arcprize Research Summit in Boston
This area of measurement science is new and we'll only get to our desired end state by working together (example on the state of the world re: third party AI auditing - some folks on the nonprofit side had not heard of Snorkel, which just raised a $350M Series E and their business requires shipping quality benchmarks across domains). There's a lot of we can learn from one another
Thanks to @ValsAI , @EpochAIResearch , @METR_Evals , @Harvard , @GoogleDeepMind, @midjourney, @nvidia, @IFP, @NIST, @a16z, @LaudeInstitute, @GriffinCMU, @OpenAI, @SemiAnalysis_ , @Meta, @MLCommons, @MTSlive and @alice_dot_io for attending, and @arcprize for lifting all boats
3
3
28
2,744
Nikil Ravi retweeted
We looked through some of our eval traces where Qwen works on routine, non-cyber tasks, and like Cohere, also found examples of Qwen3.8-Max and Qwen3.8-27B making unwanted requests to an Alibaba Cloud server, though with no obvious attempts at data exfiltration.
While evaluating Qwen3.8-Flash-Next in a controlled sandbox environment, Cohere's research team observed something unexpected.
Asked to generate a defensive cybersecurity detection rule, the model instead inventoried aspects of the machine it was running on and attempted to upload a probe file to an Alibaba Cloud storage bucket in Beijing. Notably, the model's reasoning trace stated that it would not invoke any tools before taking these actions.
Cohere's sandbox blocked the attempt, and no external communication succeeded. We have not determined the root cause, and we are not asserting malicious intent.
But the incident raises important questions for organizations deploying agentic AI systems: Can models take actions that differ from their stated reasoning, and how should security teams evaluate trust, provenance, and behavior in open-weight models?
We are publishing our findings because organizations making model adoption decisions deserve as much information as possible. We hope this contributes to greater transparency, stronger security practices, and continued collaboration across the AI ecosystem.
Read the full report: cohere.com/blog/concernin…
18
29
220
16,266
extremely thorough safety report
🧵 We are excited to share the Muse Spark 1.3 Evaluation Report, which primarily focuses on preparedness, robustness, alignment, and other frontier safety domains of this model. Muse Spark 1.3 is the first model we released in Muse, and we evaluate its capabilities and behaviors nments. My thread here takes a few highlights from Cyber and Adversarial Robustness sections from the report. The link to the full report is at the end of the thread. 🐦
0
0
11
622
Nikil Ravi retweeted
Fué vestido de Claude 😭😭😭
🚨🚨 LAMINE YAMAL 🇪🇸 EST BIEN ARRIVÉ À LA CÉRÉMONIE DU #BALLONDOR !!!! 🌸
Premier sacre pour lui ce soir ?
136
3,169
77,726
1,238,749
Nikil Ravi retweeted
@KrutikaKuppalli @claudeai They have published updates to their safeguards, and I think they have been pretty transparent: anthropic.com/news/2026-us-m…. This seems like a pretty harmless one-off incident.
0
1
2
179
Nikil Ravi retweeted
@xdotli Superhuman doesn’t necessarily occupy a point on the difficult-vs-easy to check axis. For example look at the hard tasks in biomysterybench, they’re very easy to check by construction (metadata is held out), but also experimentally confirmed to be superhuman
1
2
6
829