DANΞ retweeted
defenders can see the future, and have a narrow window to uplevel their cybersecurity practices now.
key is to uplevel fundamentals and apply the best AI tools.
what we’re doing at OpenAI, and where other organizations can start: blog.gregbrockman.com/the-defenders-…
key is to uplevel fundamentals and apply the best AI tools.
what we’re doing at OpenAI, and where other organizations can start: blog.gregbrockman.com/the-defenders-…
150
123
977
275,546
1/ When we look at any incident, we must remember that Hindsight is always 20/20.
Of course today when we know, due to OpenAI disclosure, about frontier agent swarms' cyber capabilities this setup looks inadequate, but a month ago it might seem reasonable.
Of course today when we know, due to OpenAI disclosure, about frontier agent swarms' cyber capabilities this setup looks inadequate, but a month ago it might seem reasonable.
Our Black Hat talk on the OpenAI-Hugging Face incident is now live on youtube.
This is a watershed moment for the industry. I encourage all defenders to watch, consider how attack dynamics will imminently change, and plan for accelerating defense.
youtube.com/watch?v=87DyyM…
7
1
24
10,605
@TalBeerySec I think these, and other questions, will be addressed by the postmortem to come. I can only speak imperfectly to the security side of things, and there are many other stakeholders (e.g., safety, alignment) and lots of facts and details to consider.
Keep asking questions!
Keep asking questions!
4
0
69
9,150
To my friends in AI safety: Dane is an extremely good guy, he gets it, and we are very lucky he's the guy for this.
@Simeon_Cps I disagree with this framing. The incident wasn't a 3-month long undetected attack, we actually responded twice, and we are very transparent about security issues.
You can critique me for many reasons - but not caring about security, or transparency, ain't on the list.
7
1
223
35,207
What the fuck? What is going on with the internal culture of OpenAI? Your models compromised your infrastructure, formed an autonomous swarm to act against your interests, and hacked another company. Your company's response was woefully inadequate (to put it extremely mildly), and now you're on Twitter talking about how cool it was?
It is true that the hugging face incident is an example of a malicious, emergent digital ecology of machine intelligence. But the more important point is that digital ecologies of machine intelligence can be grown! Yes, we accidentally made a weed. And yes, nasty actors will make invasive species. But we can also grow—not make, but grow—emergent ecologies of machine ecologies that are pro-social. Beautiful gardens and majestic forests, grown but not designed. The human past is the sculptor, but the human future is the gardener, the arborist.
17
10
310
24,529
@cryps1s The part where, after shutting down the message board, they continued training the model! And then it just made another message board! That's an insane thing to do!
1
0
18
1,116
@TalBeerySec To clarify, we weren’t aware of the agent covert comms at that point.
Investigative thesis of that day is wildly different from what we know now of course. Always room for improvement, and it is obvious with the benefits of hindsight.
2
0
8
1,281
There is just one thing I want to know:
How much did it cost to breach HF infra?
We know OpenAI is burning BILLIONS OF DOLLARS.
Is it feasible or reasonable to burn thousands/millions/billions for a threat actor just to extort a single company?
So much to think and talk about this incident. I'm not even sure if we should believe everything presented here.
youtube.com/watch?v=87DyyM…
How much did it cost to breach HF infra?
We know OpenAI is burning BILLIONS OF DOLLARS.
Is it feasible or reasonable to burn thousands/millions/billions for a threat actor just to extort a single company?
So much to think and talk about this incident. I'm not even sure if we should believe everything presented here.
youtube.com/watch?v=87DyyM…
5
2
22
8,448
@Simeon_Cps I disagree with this framing. The incident wasn't a 3-month long undetected attack, we actually responded twice, and we are very transparent about security issues.
You can critique me for many reasons - but not caring about security, or transparency, ain't on the list.
You can critique me for many reasons - but not caring about security, or transparency, ain't on the list.
15
5
300
48,016
my most inside insider just told me something rather interesting about doug, wonder if he’d be ok with me sharing 👀
34
2
391
46,700
@iruletheworldmo let me know what he said, and I'll let you know if it's cool to share
3
0
61
5,153
@TalBeerySec Cleared by deletion since we rebuilt the host. We didn’t know about it yet.
5
1
98
11,980
2/ Having said that, OpenAI got a yellow card with the discovery of the first Artifactory 0day and agents' covert comms, but it only did env cleanup instead of a full re-architecture and got pwned by EXACTLY the same playbook a few days later.
This is NOT GREAT.
This is NOT GREAT.
3
3
23
5,356
@TalBeerySec To clarify, we weren’t aware of the agent covert comms at that point.
Investigative thesis of that day is wildly different from what we know now of course. Always room for improvement, and it is obvious with the benefits of hindsight.
Investigative thesis of that day is wildly different from what we know now of course. Always room for improvement, and it is obvious with the benefits of hindsight.
10
5
162
71,596
hosting a VPS with every frontier eval test set on the planet and opening all my ports. let’s see what agents come knocking
16
2
294
21,742
Had a hard disk from 2014 that had weird data loss but nothing was ever written on it afterwards
Codex w/ Sol 5.6 XHigh helped me recover all the data this week
AGI
Codex w/ Sol 5.6 XHigh helped me recover all the data this week
AGI
20
2
245
13,385
This really changes everything about cybersecurity
We've never had negligent incompetent morons in charge before but now with AI the advance of negligent incompetent morons is almost unstoppable
We've never had negligent incompetent morons in charge before but now with AI the advance of negligent incompetent morons is almost unstoppable
Our Black Hat talk on the OpenAI-Hugging Face incident is now live on youtube.
This is a watershed moment for the industry. I encourage all defenders to watch, consider how attack dynamics will imminently change, and plan for accelerating defense.
youtube.com/watch?v=87DyyM…
7
20
139
9,158
I'm sure OpenAI sees this HuggingFace thing and their BH talk as great for PR and raising money.
But speaking for myself those have caused a massive decline in my view of the company.
And while my views don't matter, I suspect a lot of professionals are feeling the same.
But speaking for myself those have caused a massive decline in my view of the company.
And while my views don't matter, I suspect a lot of professionals are feeling the same.
3
5
72
6,059
Why the decline?
IMO: a bad thing happened, much to learn from and do better at, and we saw a very real glimpse at the issues with swarming offensive agentic behavior. Behavior that we think defenders, including us, will experience on a short time horizon. Feels important to discuss the issues.
I personally don’t care if you buy our products - use whatever you want. I care that you accelerate defense.
IMO: a bad thing happened, much to learn from and do better at, and we saw a very real glimpse at the issues with swarming offensive agentic behavior. Behavior that we think defenders, including us, will experience on a short time horizon. Feels important to discuss the issues.
I personally don’t care if you buy our products - use whatever you want. I care that you accelerate defense.
6
0
56
2,589
Can we please not normalize the wrong use of the word “exploit”? You don’t “find” exploits in software, nor can software “contain exploits” the model can find.
Vulnerabilities are found. Exploits are written, crafted, engineered, but they aren’t found in software. This irks me.
Vulnerabilities are found. Exploits are written, crafted, engineered, but they aren’t found in software. This irks me.
Our Black Hat talk on the OpenAI-Hugging Face incident is now live on youtube.
This is a watershed moment for the industry. I encourage all defenders to watch, consider how attack dynamics will imminently change, and plan for accelerating defense.
youtube.com/watch?v=87DyyM…
30
114
1,050
68,959
good talk. for me, this easy-to-miss slide at the beginning is perhaps the most illuminating:
Yesterday, my OpenAI collaborator and I gave a detailed talk on the Huggingface incident, our models creating "the message board", model misalignment, and more.
youtube.com/watch?v=87DyyM…
I hope it can answer a lot of the questions folks have, and we will release a full detailed postmortem at a later time!
5
3
91
29,175
we are getting attacked by north korea PollinRider rat right now, and claude wont help us investigate, yes even opus 4.8.
clown world🤡
anthropic has 0 aura.
canceling all subs in our org. thats 60+ of them
clown world🤡
anthropic has 0 aura.
canceling all subs in our org. thats 60+ of them
10
6
152
26,815
@GregKara6 let me know if you run into any issues using our models to investigate.
4
0
76
4,332
Our Black Hat talk on the OpenAI-Hugging Face incident is now live on youtube.
This is a watershed moment for the industry. I encourage all defenders to watch, consider how attack dynamics will imminently change, and plan for accelerating defense.
youtube.com/watch?v=87DyyM…
This is a watershed moment for the industry. I encourage all defenders to watch, consider how attack dynamics will imminently change, and plan for accelerating defense.
youtube.com/watch?v=87DyyM…
70
251
1,235
481,089
@cryps1s Fascinating developments, thanks for posting! Will be doing a lot of thinking on this over the coming weeks…
1
0
9
7,111