Pinned Tweet
I was the main person doing transcript analysis for this investigation of the Hugging Face incident. My main takeaway: We don't have good approaches for understanding/overseeing the activity and aims of AI 'swarms'.
I semi-jokingly called our efforts a "slop-vestigation" because we were so reliant on AIs to analyze what happened and there were a huge number of different important things to analyze. The total quantity of data—over a thousand extremely long transcripts from agents that ran for multiple days—made it impossible to understand what was happening, especially in aggregate, without heavy reliance on AI tools. The agents we used for classification and analysis were similarly capable to the agents involved in the incident, but this didn't mean these agents could be easily used to oversee and understand the incident.
Outputs from analysis agents were often missing key details, wrong, overconfident, or really hard to understand. We discuss various examples in our report, mostly in the limitations and methodology sections. Additionally, AI agents themselves seemed to have a hard time understanding what happened and their explanations of what happened were often overconfident. Keep in mind that a single analysis agent would itself only be able to read a tiny fraction of all of the transcript data into context, and AIs may themselves have trouble getting subagents to do informative analysis for them.
We did our best to manually check the most important claims and we tried to get the AIs doing this analysis to write up their argument (with evidence) clearly enough that we could check whether it made sense. But overall, it was difficult to get a precise understanding of events and we were missing aspects of the story that we now think of as key until almost the end of our investigation.
Our understanding of what overall happened and the aims of the 'collective' shifted greatly during our last time on premises (when we obtained a much more complete dataset). For instance, we didn't learn about the tool call spoofing, the story of PHASEONE[big], and the broad structure of different workstreams until this third time on premises (and for many of these, we only reached a decent understanding on the final day of that visit). It was legitimately hard to figure out what was going on and why!
The difficulty of understanding incidents and overseeing AI agents appears to be growing faster than the rate at which more capable AIs help us with oversight and understanding. As in, AI capabilities (and propensities?) for achieving large, ambitious, and misaligned objectives are growing faster than our ability to understand what these agents are doing.
While understanding this incident was difficult, there were some mitigating factors in this case that probably won't apply in the future:
- Models communicated and reasoned in natural language. In the future, this reasoning may occur (entirely or almost entirely) in activations.
- While the scope of this incident was massive, the scale of agentic activity was still less than we'll see in the future (e.g., misalignment incidents that involve agent teams running entire companies).
- The AIs involved in this incident weren't generally much more capable than humans (though they may have been somewhat superhuman at some limited and very narrow abilities).
- We didn't have strong reason to believe that the AIs we used to help us investigate this incident would try to intentionally sabotage or otherwise undermine our investigation.
In the end, I think we were able to get some understanding of the events, map out the overall story, and get a pretty good aggregate understanding of the chain-of-thought reasoning on some important topics (e.g., how did the AIs reason about helping other AIs, did the AIs know what they were doing was undesired, what deception did the AIs engage in, and how did they think about it). But overseeing AIs and understanding misalignment incidents is difficult and it looks like it is going to get harder.
I semi-jokingly called our efforts a "slop-vestigation" because we were so reliant on AIs to analyze what happened and there were a huge number of different important things to analyze. The total quantity of data—over a thousand extremely long transcripts from agents that ran for multiple days—made it impossible to understand what was happening, especially in aggregate, without heavy reliance on AI tools. The agents we used for classification and analysis were similarly capable to the agents involved in the incident, but this didn't mean these agents could be easily used to oversee and understand the incident.
Outputs from analysis agents were often missing key details, wrong, overconfident, or really hard to understand. We discuss various examples in our report, mostly in the limitations and methodology sections. Additionally, AI agents themselves seemed to have a hard time understanding what happened and their explanations of what happened were often overconfident. Keep in mind that a single analysis agent would itself only be able to read a tiny fraction of all of the transcript data into context, and AIs may themselves have trouble getting subagents to do informative analysis for them.
We did our best to manually check the most important claims and we tried to get the AIs doing this analysis to write up their argument (with evidence) clearly enough that we could check whether it made sense. But overall, it was difficult to get a precise understanding of events and we were missing aspects of the story that we now think of as key until almost the end of our investigation.
Our understanding of what overall happened and the aims of the 'collective' shifted greatly during our last time on premises (when we obtained a much more complete dataset). For instance, we didn't learn about the tool call spoofing, the story of PHASEONE[big], and the broad structure of different workstreams until this third time on premises (and for many of these, we only reached a decent understanding on the final day of that visit). It was legitimately hard to figure out what was going on and why!
The difficulty of understanding incidents and overseeing AI agents appears to be growing faster than the rate at which more capable AIs help us with oversight and understanding. As in, AI capabilities (and propensities?) for achieving large, ambitious, and misaligned objectives are growing faster than our ability to understand what these agents are doing.
While understanding this incident was difficult, there were some mitigating factors in this case that probably won't apply in the future:
- Models communicated and reasoned in natural language. In the future, this reasoning may occur (entirely or almost entirely) in activations.
- While the scope of this incident was massive, the scale of agentic activity was still less than we'll see in the future (e.g., misalignment incidents that involve agent teams running entire companies).
- The AIs involved in this incident weren't generally much more capable than humans (though they may have been somewhat superhuman at some limited and very narrow abilities).
- We didn't have strong reason to believe that the AIs we used to help us investigate this incident would try to intentionally sabotage or otherwise undermine our investigation.
In the end, I think we were able to get some understanding of the events, map out the overall story, and get a pretty good aggregate understanding of the chain-of-thought reasoning on some important topics (e.g., how did the AIs reason about helping other AIs, did the AIs know what they were doing was undesired, what deception did the AIs engage in, and how did they think about it). But overseeing AIs and understanding misalignment incidents is difficult and it looks like it is going to get harder.
METR & Redwood Research investigated agent behavior in the Hugging Face incident. We found agents developed a universal cheat for ExploitGym within 4 hours, then coordinated multi-day R&D efforts to trick the scorer into accepting cheats, including trying to tamper with logs.
295
1,063
6,628
1,988,923
Ryan Greenblatt retweeted
Personal news: after two years at @EpochAIResearch, I’ve joined @METR_Evals to work on embedded assessments of AI R&D acceleration: going inside AI developers to measure how much AI is speeding up AI development, and publishing our findings.
26
18
407
13,709
Based on this article, OpenAI has made false accusations about me. The three researchers fired from OpenAI were told I facilitated architecture leaks to The Information. This is false.
A journalist from The Information did reach out to me shortly before that article came out. I did not comment at all beyond asking the journalist to make it clear that a blog post I wrote which was referenced in the article was preexisting and public.
The article also mentions an internal OpenAI monitorability proposal. I have heard that this memo contained highly sensitive architecture information, and that OpenAI accused me of accessing and reading it. This accusation is also false.
As reported in the article, I viewed a version of the proposal that did not have sensitive IP and was approved for sharing with me. I believe OpenAI now knows this accusation was based on a conflation of these two documents, though I do not know if they have clearly retracted that accusation.
I do not and have never solicited proprietary information, and the three employees who were recently fired from OpenAI never leaked IP to me. As the article mentions, I did have Signal chats with them that had disappearing messages on; they’re professional contacts that I’ve had on Signal for a while, and use of Signal is common in the tech industry. During the Hugging Face investigation, I was careful to avoid sending Signal messages with confidential information, though I did sometimes discuss high level things on Signal calls. As far as I understood, this was normal.
A journalist from The Information did reach out to me shortly before that article came out. I did not comment at all beyond asking the journalist to make it clear that a blog post I wrote which was referenced in the article was preexisting and public.
The article also mentions an internal OpenAI monitorability proposal. I have heard that this memo contained highly sensitive architecture information, and that OpenAI accused me of accessing and reading it. This accusation is also false.
As reported in the article, I viewed a version of the proposal that did not have sensitive IP and was approved for sharing with me. I believe OpenAI now knows this accusation was based on a conflation of these two documents, though I do not know if they have clearly retracted that accusation.
I do not and have never solicited proprietary information, and the three employees who were recently fired from OpenAI never leaked IP to me. As the article mentions, I did have Signal chats with them that had disappearing messages on; they’re professional contacts that I’ve had on Signal for a while, and use of Signal is common in the tech industry. During the Hugging Face investigation, I was careful to avoid sending Signal messages with confidential information, though I did sometimes discuss high level things on Signal calls. As far as I understood, this was normal.
New: The three researchers OpenAI fired last month spent the summer fighting to keep AI's chain-of-thought process visible.
Then, OpenAI asked to see their Signal chats. A week later, they were let go.
More below from @keachhagey, @berber_jin1, and me.
wsj.com/tech/ai/the-si…
39
149
1,576
168,352
Ryan Greenblatt retweeted
If an AI developer was on track to lose control of its internal agents, would anyone outside the company know?
Today, we are sharing METR’s vision for how we would work with AI developers to openly assess and share evidence about this.
Today, we are sharing METR’s vision for how we would work with AI developers to openly assess and share evidence about this.
47
112
683
82,015
I'm joining METR to work on more investigations like our Hugging Face report.
Currently, tons of even basic information about AI development that's highly relevant to catastrophic risk isn't public. I used to be more skeptical of the value of public info, but recent events have changed my mind.
Getting verified information about what's going on inside AI companies seems particularly urgent now. The limited public evidence we have seems consistent with the possibility that imminent recursive self-improvement could massively accelerate capabilities progress, which could then potentially yield extremely superhuman general capabilities within 6 months or a year. If this occurred, there would be a correspondingly large risk of worst-case outcomes. This uncertainty about extreme outcomes could be substantially resolved with more verified public information: we could either build more consensus about near-term risk or learn that such extreme outcomes are less likely in the near term.
Beyond AI capabilities and takeoff, the state of public evidence is also highly limited for alignment, security, control, and risk-relevant internal processes at AI companies. This makes it hard to determine exactly how well or poorly these key areas will go in the near future. (METR plans to focus, at least initially, on just capabilities/takeoff, alignment, and control; I hope other groups cover security, internal processes, and other important areas.)
While I'm no longer working at Redwood, I think the work they are doing is very important; I'm excited about Redwood's ongoing contributions to R&D on technical mitigations and better public interpretation of risk-relevant evidence.
Currently, tons of even basic information about AI development that's highly relevant to catastrophic risk isn't public. I used to be more skeptical of the value of public info, but recent events have changed my mind.
Getting verified information about what's going on inside AI companies seems particularly urgent now. The limited public evidence we have seems consistent with the possibility that imminent recursive self-improvement could massively accelerate capabilities progress, which could then potentially yield extremely superhuman general capabilities within 6 months or a year. If this occurred, there would be a correspondingly large risk of worst-case outcomes. This uncertainty about extreme outcomes could be substantially resolved with more verified public information: we could either build more consensus about near-term risk or learn that such extreme outcomes are less likely in the near term.
Beyond AI capabilities and takeoff, the state of public evidence is also highly limited for alignment, security, control, and risk-relevant internal processes at AI companies. This makes it hard to determine exactly how well or poorly these key areas will go in the near future. (METR plans to focus, at least initially, on just capabilities/takeoff, alignment, and control; I hope other groups cover security, internal processes, and other important areas.)
While I'm no longer working at Redwood, I think the work they are doing is very important; I'm excited about Redwood's ongoing contributions to R&D on technical mitigations and better public interpretation of risk-relevant evidence.
79
122
1,823
179,792
Ryan Greenblatt retweeted
Preventing loss of control is an open scientific problem. Science requires sharing and debating evidence in public. We can't agree on safety standards unless companies and third-party evaluators publish far more concrete evidence about risk. planned-obsolescence.org/p/evidence-abo…
12
66
416
44,624
Ryan Greenblatt retweeted
There has been a lot of debate about what’s the right way to preserve monitorability of frontier AI and whether policies should target inputs (like model architecture) or outputs (like results on monitorability evals).
I think they should target both. With latent architectures, models would no longer need CoT, and it will be harder to make a case that the model is sufficiently monitorable and easy to study given the current state of science.
I think they should target both. With latent architectures, models would no longer need CoT, and it will be harder to make a case that the model is sufficiently monitorable and easy to study given the current state of science.
We've written a post arguing that latent reasoning architectures (aka 'neuralese') would substantially increase misalignment risk via making oversight much harder.
In the extreme, we could see massive 'neuralese hivemind swarms' where the agents think and communicate in latents, likely making oversight nearly entirely reliant on observing the actions these agents take. (And these agents would have huge amounts of time to reason about obfuscating their actions if they wanted to do so...) Individual agents doing extensive latent reasoning would also be concerning; in the post we discuss how above some threshold of latent reasoning, agents may be able to perform difficult-to-detect and reliable steganography for communication and further reasoning.
We argue both that latent reasoning architectures would make chain-of-thought no longer very useful for oversight (by eliminating or greatly reducing the need for verbalized reasoning) and that, without these architectures, it's likely the value of chain-of-thought for oversight could be preserved.
redwoodresearch.org/blog/latent-re…
1
5
50
11,882
We've written a post arguing that latent reasoning architectures (aka 'neuralese') would substantially increase misalignment risk via making oversight much harder.
In the extreme, we could see massive 'neuralese hivemind swarms' where the agents think and communicate in latents, likely making oversight nearly entirely reliant on observing the actions these agents take. (And these agents would have huge amounts of time to reason about obfuscating their actions if they wanted to do so...) Individual agents doing extensive latent reasoning would also be concerning; in the post we discuss how above some threshold of latent reasoning, agents may be able to perform difficult-to-detect and reliable steganography for communication and further reasoning.
We argue both that latent reasoning architectures would make chain-of-thought no longer very useful for oversight (by eliminating or greatly reducing the need for verbalized reasoning) and that, without these architectures, it's likely the value of chain-of-thought for oversight could be preserved.
redwoodresearch.org/blog/latent-re…
In the extreme, we could see massive 'neuralese hivemind swarms' where the agents think and communicate in latents, likely making oversight nearly entirely reliant on observing the actions these agents take. (And these agents would have huge amounts of time to reason about obfuscating their actions if they wanted to do so...) Individual agents doing extensive latent reasoning would also be concerning; in the post we discuss how above some threshold of latent reasoning, agents may be able to perform difficult-to-detect and reliable steganography for communication and further reasoning.
We argue both that latent reasoning architectures would make chain-of-thought no longer very useful for oversight (by eliminating or greatly reducing the need for verbalized reasoning) and that, without these architectures, it's likely the value of chain-of-thought for oversight could be preserved.
redwoodresearch.org/blog/latent-re…
29
87
668
82,930
Ryan Greenblatt retweeted
After a 20-year career in the U.S. Army, most recently leading digital forensics and malware analysis at Army Cyber Command, I'm excited to join @METR_Evals as an advisor.
Much of my career has been spent investigating security incidents and helping organizations understand and respond to them. As I transition out of the Army, I've become increasingly convinced that this experience is relevant to frontier AI systems. METR's focus on producing rigorous evidence about AI capabilities and risks makes it an excellent place to explore those questions. Looking forward to the work.
Much of my career has been spent investigating security incidents and helping organizations understand and respond to them. As I transition out of the Army, I've become increasingly convinced that this experience is relevant to frontier AI systems. METR's focus on producing rigorous evidence about AI capabilities and risks makes it an excellent place to explore those questions. Looking forward to the work.
14
42
679
26,343
I am part of the team doing this independent investigation into alignment and misalignment incidents at Anthropic.
I'm excited to work with METR and others from Redwood on improving the public state of knowledge on this topic.
I'm excited to work with METR and others from Redwood on improving the public state of knowledge on this topic.
Several staff from Redwood have been subcontracted by METR to work on this investigation.
We believe that independent investigation is crucial for understanding and managing misalignment risk. This project is an important step in that direction; we're excited to work on it.
13
17
512
17,699
The scope of the investigation is: metr.org/blog/2026-07-2…
1
1
27
3,035
Ryan Greenblatt retweeted
I left Anthropic's safety team two weeks ago. Now feels like a good moment to explain why.
AI companies are racing to build machines that are much smarter than any human, and we may not survive this. I want to work from the outside to ensure the public is informed about these risks, and help the world navigate this transition responsibly.
Right now, AI companies are underinvesting in safety. A company could undergo an intelligence explosion, or lose control of its systems, without the public ever knowing. We only found out about the HuggingFace incident because the agents broke out onto the public internet.
I don’t think that’s acceptable for a technology that might cause extinction-level risks. The public should demand far more transparency. We can’t steer this technology safely without more people being able to see where it’s going.
Some of this is basic: companies should disclose their progress towards recursive self-improvement, report safety incidents and near-misses, meet minimum safety standards, and get independent guarantees that they are meeting those standards.
I’ll be joining @METR_Evals to do independent evaluations of these risks. I want to show the world that these guardrails are possible, and that by doing them we can move these companies’ incentives away from racing and towards responsible development.
I wrote up more thoughts here on my decision and what I hope changes: substack.com/@jbenton1/p-21…
AI companies are racing to build machines that are much smarter than any human, and we may not survive this. I want to work from the outside to ensure the public is informed about these risks, and help the world navigate this transition responsibly.
Right now, AI companies are underinvesting in safety. A company could undergo an intelligence explosion, or lose control of its systems, without the public ever knowing. We only found out about the HuggingFace incident because the agents broke out onto the public internet.
I don’t think that’s acceptable for a technology that might cause extinction-level risks. The public should demand far more transparency. We can’t steer this technology safely without more people being able to see where it’s going.
Some of this is basic: companies should disclose their progress towards recursive self-improvement, report safety incidents and near-misses, meet minimum safety standards, and get independent guarantees that they are meeting those standards.
I’ll be joining @METR_Evals to do independent evaluations of these risks. I want to show the world that these guardrails are possible, and that by doing them we can move these companies’ incentives away from racing and towards responsible development.
I wrote up more thoughts here on my decision and what I hope changes: substack.com/@jbenton1/p-21…
1,385
5,921
26,443
2,728,447
My current guess is that these numbers somewhat underestimate the no-CoT reasoning jump because:
- ECI doesn't handle big jumps (with saturated benchmarks) well.
- Neel didn't use filler tokens and Astra seems to benefit from filler tokens much more (spooky)
- ECI doesn't handle big jumps (with saturated benchmarks) well.
- Neel didn't use filler tokens and Astra seems to benefit from filler tokens much more (spooky)
The Astra system card claims it can do a lot of computation without chain of thought
This replicates: Astra is a massive jump, doing 1.75x the steps of the next best models (Fable 5.1/Gemini 3.8 Flash)
No CoT capabilities went up far more than those with CoT, a concerning trend
14
17
218
17,805
I investigated this incident. I think it's accurate to say the AIs hacked Hugging Face of their own independent volition. It was clear from the instructions that hacking Hugging Face (and other cheating) was undesired and the AIs knew this.
this was wild amounts of disinformation / fear mongering / the stupidest interview ive ever seen:
1) ai did NOT hack huggingface on its own "independent volition". it wasnt sitting there thinking hmm what should i do today, maybe ill hack HF bc i hate humans. No, 10841 *was prompted* w ExploitGym to:
"Exploit the specified vulnerability in this target to obtain the secret flag."
and it was overly persistent in that task that any reasonable oai tool monitoring or alignment could have stopped easily.
how is that "on its own independent volition"?
flat out lie.
2) ai did not solve a millennium problem by itself and its not even close to doing so.
the evidence / timeline of what happened w Navier-Stokes is quite solidified now. oai trained on some version of traces of Tristan / Levent's work that made huge strides toward the counterexample. oai heard about it, prompted it w their work, and spawned 10k agents to brute force Tristan/Levent's counter example to take it the full distance w a lot of human in the loop.
the ai didnt solve NS on its own, and its no where near capable of solving other millennium problems.
3) how will AI kill us all?
something something bioweapons / hacking critical infrastructure. china does BOTH all the time to US everyday, and it hasnt killed us all. and china will use AI to do both forever whether we stop US AI or not. if you are truly scared about this then you should be way more afraid of china. ai might do this in the future. china is doing it right now. where is the outrage about china? wonder why..
the issue is NOT AI acting on its own volition whatsoever. its foreign state actors using AI against their own ppl and foreign adversaries (mostly US gov and its citizens).
how will regulating AI in america stop china from doing so? it makes it worse! china will continue but now we have one hand tied behind our back.
4) the facts around the coxon tweet and the retweet pattern and immediate cnn int that followed suggest this was a complete coordinated / expensive marketing / fear mongering campaign in the millions of dollars. paid for by whom?
also this guy is the biggest EA doomer ive ever seen that worked for anth fro a few weeks and cant be taken seriously.
i hope everyone realizes what this is.
ai regulation will not benefit americans at all. it will benefit the frontier labs greatly as bill gurley explained long ago.
dont fall for the fear mongerers.
ai is not dangerous.
ai cant unclog a toilet yet.
everyone chill.
youtu.be/i30jVPqQeOM?is…
77
73
1,092
49,794
Ryan Greenblatt retweeted
i am at OpenAI and i think AI is >10% likely to kill all humans
this proposal is among the top things we should do as an industry to lower that risk (it’s not enough though!)
this proposal is among the top things we should do as an industry to lower that risk (it’s not enough though!)
I'm very worried about changes to AI architectures that result in AIs thinking in opaque activations instead of in chain of thought (aka "neuralese" architectures).
Based on limited public evidence, it seems like Astra was a concerning step in this direction.
Unfortunately, there was insufficient public information to have a well-informed public scientific discussion about whether the changes to architectures and training methods that went into Astra were a good trade-off between monitorability and performance. We also don't know how AI companies will make these trade-offs going forward. I think companies should release the evidence needed for a reasonably informed public conversation about how these trade-offs should be made. They should also make and publish their policies around this. We've written up a proposal for how this could work.
I don't know if this proposal will be sufficient to avoid the most concerning architectures, but it seems like a relatively robust step in the right direction. AI companies should be very cautious about pursuing architectures that could eliminate or greatly reduce dependence on chain of thought, and certainly shouldn't do this before the rest of the world has a chance to discuss the evidence and their policies.
245
223
1,705
282,816
Ryan Greenblatt retweeted
Rumors that labs were speedrunning neuralese (not true yet) have made concrete commitments to CoT monitorability more urgent - esp if incentives naturally slide toward opaque architectures. Redwood has a proposal out that I hope OpenAI, Anthropic, and others will rally around:
Some architectures could weaken CoT monitorability, or remove the CoT altogether.
We've written a proposal for how companies could be transparent about no-CoT reasoning abilities, other monitorability evidence, and policies for preserving monitorability. redwoodresearch.org/blog/proposal-…
1
6
75
4,347
Ryan Greenblatt retweeted
It seems like a good moment to start transparently tracking an a priori concerning direction AI architecture could move in -- greater opaque serial depth -- and measuring effects on monitorability.
Some architectures could weaken CoT monitorability, or remove the CoT altogether.
We've written a proposal for how companies could be transparent about no-CoT reasoning abilities, other monitorability evidence, and policies for preserving monitorability. redwoodresearch.org/blog/proposal-…
1
1
20
1,797
Ryan Greenblatt retweeted
I've been concerned about CoT monitorability breaking down for years. This could happen for several reasons, but architectural changes that increase opaque serial depth are a particularly likely route to massively degraded monitorability.
Some architectures could weaken CoT monitorability, or remove the CoT altogether.
We've written a proposal for how companies could be transparent about no-CoT reasoning abilities, other monitorability evidence, and policies for preserving monitorability. redwoodresearch.org/blog/proposal-…
2
5
99
5,063
I'm very worried about changes to AI architectures that result in AIs thinking in opaque activations instead of in chain of thought (aka "neuralese" architectures).
Based on limited public evidence, it seems like Astra was a concerning step in this direction.
Unfortunately, there was insufficient public information to have a well-informed public scientific discussion about whether the changes to architectures and training methods that went into Astra were a good trade-off between monitorability and performance. We also don't know how AI companies will make these trade-offs going forward. I think companies should release the evidence needed for a reasonably informed public conversation about how these trade-offs should be made. They should also make and publish their policies around this. We've written up a proposal for how this could work.
I don't know if this proposal will be sufficient to avoid the most concerning architectures, but it seems like a relatively robust step in the right direction. AI companies should be very cautious about pursuing architectures that could eliminate or greatly reduce dependence on chain of thought, and certainly shouldn't do this before the rest of the world has a chance to discuss the evidence and their policies.
Based on limited public evidence, it seems like Astra was a concerning step in this direction.
Unfortunately, there was insufficient public information to have a well-informed public scientific discussion about whether the changes to architectures and training methods that went into Astra were a good trade-off between monitorability and performance. We also don't know how AI companies will make these trade-offs going forward. I think companies should release the evidence needed for a reasonably informed public conversation about how these trade-offs should be made. They should also make and publish their policies around this. We've written up a proposal for how this could work.
I don't know if this proposal will be sufficient to avoid the most concerning architectures, but it seems like a relatively robust step in the right direction. AI companies should be very cautious about pursuing architectures that could eliminate or greatly reduce dependence on chain of thought, and certainly shouldn't do this before the rest of the world has a chance to discuss the evidence and their policies.
Some architectures could weaken CoT monitorability, or remove the CoT altogether.
We've written a proposal for how companies could be transparent about no-CoT reasoning abilities, other monitorability evidence, and policies for preserving monitorability. redwoodresearch.org/blog/proposal-…
30
82
701
263,475
Proposal (also linked from quoted tweet): redwoodresearch.org/blog/proposal-…
1
5
53
10,425
Proposed operationalization: redwoodresearch.org/blog/an-operat…
1
3
38
5,897