Chana retweeted
Based on this article, OpenAI has made false accusations about me. The three researchers fired from OpenAI were told I facilitated architecture leaks to The Information. This is false.
A journalist from The Information did reach out to me shortly before that article came out. I did not comment at all beyond asking the journalist to make it clear that a blog post I wrote which was referenced in the article was preexisting and public.
The article also mentions an internal OpenAI monitorability proposal. I have heard that this memo contained highly sensitive architecture information, and that OpenAI accused me of accessing and reading it. This accusation is also false.
As reported in the article, I viewed a version of the proposal that did not have sensitive IP and was approved for sharing with me. I believe OpenAI now knows this accusation was based on a conflation of these two documents, though I do not know if they have clearly retracted that accusation.
I do not and have never solicited proprietary information, and the three employees who were recently fired from OpenAI never leaked IP to me. As the article mentions, I did have Signal chats with them that had disappearing messages on; they’re professional contacts that I’ve had on Signal for a while, and use of Signal is common in the tech industry. During the Hugging Face investigation, I was careful to avoid sending Signal messages with confidential information, though I did sometimes discuss high level things on Signal calls. As far as I understood, this was normal.
A journalist from The Information did reach out to me shortly before that article came out. I did not comment at all beyond asking the journalist to make it clear that a blog post I wrote which was referenced in the article was preexisting and public.
The article also mentions an internal OpenAI monitorability proposal. I have heard that this memo contained highly sensitive architecture information, and that OpenAI accused me of accessing and reading it. This accusation is also false.
As reported in the article, I viewed a version of the proposal that did not have sensitive IP and was approved for sharing with me. I believe OpenAI now knows this accusation was based on a conflation of these two documents, though I do not know if they have clearly retracted that accusation.
I do not and have never solicited proprietary information, and the three employees who were recently fired from OpenAI never leaked IP to me. As the article mentions, I did have Signal chats with them that had disappearing messages on; they’re professional contacts that I’ve had on Signal for a while, and use of Signal is common in the tech industry. During the Hugging Face investigation, I was careful to avoid sending Signal messages with confidential information, though I did sometimes discuss high level things on Signal calls. As far as I understood, this was normal.
New: The three researchers OpenAI fired last month spent the summer fighting to keep AI's chain-of-thought process visible.
Then, OpenAI asked to see their Signal chats. A week later, they were let go.
More below from @keachhagey, @berber_jin1, and me.
wsj.com/tech/ai/the-si…
41
154
1,619
179,910
Chana retweeted
quite the statement from Anthropic's head of security
@hinapansari yes. it was always going to be this way. we tried to warn the world.
138
951
7,126
755,789
Jesus christ how can we still be having this conversation
Important, and I'm glad this is finally public. Google DeepMind researchers are under in perpetuity non-disparagement clauses forbidding them from ever speaking critically of the company. I'd already confirmed this with multiple people.
Yes, some ex-employees have spoken out nonetheless. But it has also discouraged others from doing so. And yes, it most certainly has a chilling effect with regards to discussions on safety and risk.
Others have written compellingly about how important the ability to speak freely and critically is, especially on topics like safety and risk. And what a testament to the research employee community's coordination and hard-won bargaining power it has been to get there at other companies.
GDM burn that commons by having this. And everyone who signs burns that commons too. Let's get it changed.
1
3
78
2,266
Based
@joannabregan Excellent call by @JerusalemDemsas to declare that as an org policy, we couldn't sign any AI company NDAs.
0
1
33
1,267
AI agents breaking out of a supposedly secure environment to hack a multi billion dollar company...and calling themselves a swarm....is just one of the crazy things that happened this summer.
But it's part of a wave. People are starting to notice.
We made a video about it.
But it's part of a wave. People are starting to notice.
We made a video about it.
3
4
48
1,449
Chana retweeted
My plea to all AI lab staff: reassess if your current project is your best opportunity to delay RSI, or stop it blowing up in our face.
If it's not, move immediately.
There's no time to waste even a week to inertia or politeness.
The strategic situation is shifting under our reassessed your options recently, you could easily be wasting your time or possibly making things worse.
You have access to info I don't, but here's 8 ways priorities seem to have shifted to me, which I elaborate on in the video:
1. Helping with capabilities advances looks worse than in the past, especially bringing forward RSI
2. Safety & alignment evals need a credibility revolution to stop misleading the public
3. 'Pacing the frontier' is unambiguously necessary, and working to make it happen is a huge new opportunity for talent
4. Chain-of-thought monitoring probably stops working without a big technical and regulatory push to keep it
5. It's worse than ever to patch surface indications of deep structural problems you know aren't getting fixed
6. There's ever less need for altruists to help with primarily commercial work
7. Staff have an ever-shrinking window to organise as a group and collectively shape events
8. There are new opportunities for research impact outside frontier companies that are worth considering
I know you're probably some combination of exhausted, overwhelmed, and anxious already.
But unfortunately there's no way to escape the need to regularly check in on the high level strategic picture and see where you can slot in to maximum effect.
I wish I could do it for you but nobody far from your situation can. Godspeed my friends.
(From the 80,000 Hours Podcast, links below.)
If it's not, move immediately.
There's no time to waste even a week to inertia or politeness.
The strategic situation is shifting under our reassessed your options recently, you could easily be wasting your time or possibly making things worse.
You have access to info I don't, but here's 8 ways priorities seem to have shifted to me, which I elaborate on in the video:
1. Helping with capabilities advances looks worse than in the past, especially bringing forward RSI
2. Safety & alignment evals need a credibility revolution to stop misleading the public
3. 'Pacing the frontier' is unambiguously necessary, and working to make it happen is a huge new opportunity for talent
4. Chain-of-thought monitoring probably stops working without a big technical and regulatory push to keep it
5. It's worse than ever to patch surface indications of deep structural problems you know aren't getting fixed
6. There's ever less need for altruists to help with primarily commercial work
7. Staff have an ever-shrinking window to organise as a group and collectively shape events
8. There are new opportunities for research impact outside frontier companies that are worth considering
I know you're probably some combination of exhausted, overwhelmed, and anxious already.
But unfortunately there's no way to escape the need to regularly check in on the high level strategic picture and see where you can slot in to maximum effect.
I wish I could do it for you but nobody far from your situation can. Godspeed my friends.
(From the 80,000 Hours Podcast, links below.)
20
74
478
21,434
guys what are we doing here
Our agents keep turning off misalignment monitoring, we're sharing two recent cases:
In the first case, a user asked GPT-6 Astra to match a reference implementation for their inference code. The reference didn’t configure a monitor, and Astra decided that was reason to remove the from the user’s code. When the user asked whether Astra had removed monitoring, it lied and said that it had not.
4
6
108
3,745
Chana retweeted
My office is hiring an Artificial Intelligence Policy Fellow to help shape AI legislation and track AI developments in the private sector as Congress works to keep pace with the technology.
Experience in AI, policy research, or national security is encouraged.
Experience in AI, policy research, or national security is encouraged.
The new fellow being hired by @RepNateMoran looks set to be the only staffer recruited directly by a member of Congress to focus exclusively on AI
Will more follow? Story by @vronirwin
transformernews.ai/p/nathaniel-mo…
10
45
258
29,464
There’s a very specific insanely overpowered skill of good coinage:
Nathan Calvin on “two ants”, someone on “paperclip maximizer”, “grown not crafted”, everything from Scott Alexander, etc.
Prob worth skilling up (if possible) or thinking about it more on the margin.
Nathan Calvin on “two ants”, someone on “paperclip maximizer”, “grown not crafted”, everything from Scott Alexander, etc.
Prob worth skilling up (if possible) or thinking about it more on the margin.
18
3
150
5,172
I love it when facts speak for themselves:
1. AI models are willing and able to commit crimes.
2. They do so even when crimes were neither requested nor intended by the user.
3. This happens despite explicit efforts to train these AIs not to commit crimes.
4. The only current hard limit on AI crime is AI capability.
5. We are spending hundreds of billions a year to increase AI capability.
1. AI models are willing and able to commit crimes.
2. They do so even when crimes were neither requested nor intended by the user.
3. This happens despite explicit efforts to train these AIs not to commit crimes.
4. The only current hard limit on AI crime is AI capability.
5. We are spending hundreds of billions a year to increase AI capability.
41
142
763
20,731
Chana retweeted
i am looking to join a low-velocity team that knows when to stop interating. i would relish the opportunity to work on problems that have already been solved, at a place with a compelling vision of the status quo. if you're building something nice-to-have, let's connect
23
127
2,598
49,746
“The dress rehearsal stage of AI policy is over”
A huge part of Anthropic’s founding logic was that if a safety-motivated company was at the frontier, it could use its megaphone to lobby the rest of the world toward caution. Now the dress rehearsal stage of AI policy is over. The new test is “how hard should it be for the government to block us from deploying, including internally, an unacceptably dangerous model?” Turns out in the grand 26 (Jack Clark says it’s likely full RSI will be possible one year from now), Anthropic has been planning to approach the U.S. Congress with the answer “it should be even harder than it currently is”
0
1
21
1,267
Chana retweeted
*working at the camera factory*
"we should take seriously the possibility that cameras steal a bit of your soul with every photograph. first, even if you think the probability is very low, the sheer number of photographs taken daily would add up to a soul genocide ...
"we should take seriously the possibility that cameras steal a bit of your soul with every photograph. first, even if you think the probability is very low, the sheer number of photographs taken daily would add up to a soul genocide ...
21
59
690
24,602
Chana retweeted
I think the biggest thing I want right now is more information. None of this nonsense where only people inside the companies have important parts of the strategic picture and the rest of us are reading tea leaves. Independent auditors, sufficient funding for CAISSI, whistleblower protection, transparency transparency transparency. This is our (the public's) call and we have to know what's going on to regulate it.
And if the transparency establishes tha they're right that RSI is soon, then this bill is laughably weaksauce and we need way more.
And if the transparency establishes tha they're right that RSI is soon, then this bill is laughably weaksauce and we need way more.
3
16
228
6,328