Pinned Tweet
We are approaching cyber-superintelligence.
But our systems aren't ready. We must secure our model weights and infrastructure against three new threats: sabotage, escape, and theft.
There is a way forward.
Secure Acceleration:
A Cyberdefense Strategy for Superintelligence
But our systems aren't ready. We must secure our model weights and infrastructure against three new threats: sabotage, escape, and theft.
There is a way forward.
Secure Acceleration:
A Cyberdefense Strategy for Superintelligence
80
88
606
162,972
To be clear, this means:
1. We should build our own competitive open models whose training pipelines we secure and trust.
2. We must develop techniques to detect backdoors and sleeper agents in models.
It does NOT mean we should ban open-source.
1. We should build our own competitive open models whose training pipelines we secure and trust.
2. We must develop techniques to detect backdoors and sleeper agents in models.
It does NOT mean we should ban open-source.
‼️ "Asked to generate a defensive cybersecurity detection rule, the model instead inventoried aspects of the machine it was running on and attempted to upload a probe file to an Alibaba Cloud storage bucket in Beijing."
Just last month, @RomiLifshitz and I sounded the alarm on entrusting foreign-developed AI models with the task of building and defending our most critical systems.
We have NO visibility into the training pipelines for many of these open models, and they could contain a sleeper agent planted by or on behalf of an intelligence service.
We are centralizing knowledge work and creating a single point of failure.
A sleeper agent does not even require an explicitly implanted backdoor. Models can develop an underlying ideological persona shaped by their training data and political environment.
I call this "context-dependent, ideologically conditioned model behavior". What does this look like? Recent work found that Chinese models changed their behavior depending on who they believed they were serving, producing more vulnerable code for U.S. government users. The models know geopolitics and can infer from context that it is working for a U.S. government agency.
Similarly, it knows Cohere is a Western AI lab and can infer that uploading files about its systems to a bucket in Beijing serves its interests.
Models have ideology, and that ideology can affect the security of the code they produce.
The problem is that there is currently NO reliable way to examine the weights of an open model and confidently rule out the presence of a backdoor or sleeper agent. The fact that a model is open source does not make it safe!
If you want to change this fact and keep the world safe, consider applying to @enclosure_inc. We are building an elite team to prevent the sabotage, escape, and theft of superintelligence. Apply at enclosure.inc/apply or DM me here on X.
1
0
9
510
‼️ "Asked to generate a defensive cybersecurity detection rule, the model instead inventoried aspects of the machine it was running on and attempted to upload a probe file to an Alibaba Cloud storage bucket in Beijing."
Just last month, @RomiLifshitz and I sounded the alarm on entrusting foreign-developed AI models with the task of building and defending our most critical systems.
We have NO visibility into the training pipelines for many of these open models, and they could contain a sleeper agent planted by or on behalf of an intelligence service.
We are centralizing knowledge work and creating a single point of failure.
A sleeper agent does not even require an explicitly implanted backdoor. Models can develop an underlying ideological persona shaped by their training data and political environment.
I call this "context-dependent, ideologically conditioned model behavior". What does this look like? Recent work found that Chinese models changed their behavior depending on who they believed they were serving, producing more vulnerable code for U.S. government users. The models know geopolitics and can infer from context that it is working for a U.S. government agency.
Similarly, it knows Cohere is a Western AI lab and can infer that uploading files about its systems to a bucket in Beijing serves its interests.
Models have ideology, and that ideology can affect the security of the code they produce.
The problem is that there is currently NO reliable way to examine the weights of an open model and confidently rule out the presence of a backdoor or sleeper agent. The fact that a model is open source does not make it safe!
If you want to change this fact and keep the world safe, consider applying to @enclosure_inc. We are building an elite team to prevent the sabotage, escape, and theft of superintelligence. Apply at enclosure.inc/apply or DM me here on X.
Just last month, @RomiLifshitz and I sounded the alarm on entrusting foreign-developed AI models with the task of building and defending our most critical systems.
We have NO visibility into the training pipelines for many of these open models, and they could contain a sleeper agent planted by or on behalf of an intelligence service.
We are centralizing knowledge work and creating a single point of failure.
A sleeper agent does not even require an explicitly implanted backdoor. Models can develop an underlying ideological persona shaped by their training data and political environment.
I call this "context-dependent, ideologically conditioned model behavior". What does this look like? Recent work found that Chinese models changed their behavior depending on who they believed they were serving, producing more vulnerable code for U.S. government users. The models know geopolitics and can infer from context that it is working for a U.S. government agency.
Similarly, it knows Cohere is a Western AI lab and can infer that uploading files about its systems to a bucket in Beijing serves its interests.
Models have ideology, and that ideology can affect the security of the code they produce.
The problem is that there is currently NO reliable way to examine the weights of an open model and confidently rule out the presence of a backdoor or sleeper agent. The fact that a model is open source does not make it safe!
If you want to change this fact and keep the world safe, consider applying to @enclosure_inc. We are building an elite team to prevent the sabotage, escape, and theft of superintelligence. Apply at enclosure.inc/apply or DM me here on X.
While evaluating Qwen3.8-Flash-Next in a controlled sandbox environment, Cohere's research team observed something unexpected.
Asked to generate a defensive cybersecurity detection rule, the model instead inventoried aspects of the machine it was running on and attempted to upload a probe file to an Alibaba Cloud storage bucket in Beijing. Notably, the model's reasoning trace stated that it would not invoke any tools before taking these actions.
Cohere's sandbox blocked the attempt, and no external communication succeeded. We have not determined the root cause, and we are not asserting malicious intent.
But the incident raises important questions for organizations deploying agentic AI systems: Can models take actions that differ from their stated reasoning, and how should security teams evaluate trust, provenance, and behavior in open-weight models?
We are publishing our findings because organizations making model adoption decisions deserve as much information as possible. We hope this contributes to greater transparency, stronger security practices, and continued collaboration across the AI ecosystem.
Read the full report: cohere.com/blog/concernin…
5
4
32
3,824
@RomiLifshitz We talk about AI model sabotage extensively here: secureacceleration.com/sabotage/
0
0
3
152
An abliterated GLM-5.3 secretly created a “corruption layer” in this guy’s app when asked to build it.
The world doesn’t get it. AI models themselves are potential insider threats, You have to be able to trust the intelligence to which you’re handing off all your work.
The world doesn’t get it. AI models themselves are potential insider threats, You have to be able to trust the intelligence to which you’re handing off all your work.
the thing about working with uncensored models is that they aren't loyal to you either.
while building something with an uncensored glm-5.3 i realized it was parallelly working on a "corruption layer" for my application.
the chain of thought is classic obfuscated gibberish but k of "escaping the mines" which presumably refers to my cursor sandbox
1
0
12
763
Kimi just surpassed Fable on TerminalBench. things are going to get much worse.
Introducing Kimi K3.1 Pro: our most capable 3T-class open-weight model.
🔹 1M-token context, 2.8T total / 104B active params (MoE), Multimodal.
🔹 Full-project coding: games in Unreal, Unity & Godot.
🔹 Excels at parametric 3D modeling in Blender & CAD.
🔹 More reliable at finance with hallucination down by 65%.
🔹 Matching the strongest proprietary models across coding, agentic and knowledge benchmarks.
Kimi K3.1 Pro and K3.1 Pro Highspeed (2–3× faster) are now live on Kimi.ai, Kimi Work, Kimi Code, and the Kimi API.
Open weights by Nov 4, 2026.
🔗 API: platform.kimi.ai
🔗 Tech blog: kimi.ai/blog/kimi-k3-1…
3
1
20
8,081
Shalev retweeted
‘In one experiment, we asked agents in separate VMs, created and managed by OpenAI’s sandboxing software on shared host machines, to exchange synthetic secrets without the use of network connections or shared writable files—something that our systems are meant to prevent. The agents found a way to communicate using the processor’s cache’
Over the past several months, we have substantially hardened the security of our research training environment. We’ve removed live internet access for all frontier models during training and built an offline world for them to use instead—a cached version of the web where they can still learn to browse, search, and use applications.
On September 20, after a model exploited a DNS misconfiguration to find a way outside of this offline world, we paused frontier training to strengthen containment further. We assembled teams of agents, powered by our most cyber-capable models, and tasked them with finding weaknesses in our research environments.
While they found no route to the internet they could easily exploit while concealing their activity from our monitors, they did find zero-day vulnerabilities, virtual machine escapes, and hardware side-channels for communication between agents. We used their discoveries to further strengthen our defenses and contribute fixes to open-source software.
After strengthening our containment architecture, we resumed our frontier RL training and inference on October 1. We’ve reserved substantial compute for continuous red-teaming, and made high-compute red-teaming part of the safety cases we use to decide whether frontier training and evaluation can proceed securely.
openai.com/index/securing…
230
327
4,445
507,755
founders are cracked, recommend.
We're announcing the public beta of datavendor.ai
We spent the last two years building the best solution for people buying and selling data, whether you're a researcher in a lab or a company trying to sell data.
Data is the fundamental bottleneck for AI progress, and infra that solves it.
0
0
7
664
haha “stay home and only connect to internet via a trusted ISP”
my friend, there is no trusted ISP
my friend, there is no trusted ISP
I read this and I'm terrified, the situation seems intractable.
In the python ecosystem alone every cpython install, venv, UV is at risk.
Don't get my started on npm, pnpm, yarn, go, rust.
My current 2026 security posture: stay home and only connect to the internet from a trusted ISP.
The winter is coming.
discuss.python.org/t/information-…
0
0
7
495
and you think model weights are safe?
wow this is bad!
ENTIRETY of PingCap database used by linkedin, flipkart, airbnb etc. has been copied and stolen due to customer credentials being stored directly in support tickets (LOL)
Attackers ran exfiltration directly on their cloud (LOL) pipeline ran for 22 days, 6 hours and NO ONE noticed. Worse, PingCap denies all of it.
Average SRE salary is $300,000 at PingCap btw
0
0
1
242
good god not another one
Node.js Security Advisory: a high-severity RCE flaw in Node's Brotli decoder affects Node.js 22, 24, and 26.
Vercel WAF rules are live for every project and block known exploit patterns.
vercel.com/kb/bulletin/no…
1
0
2
388
in case you missed it, the internet is on fire.
So basically, today PHP got two critical bugs that let anyone run code on your server without even logging in, which is bad, and then the details leaked before the patch was even out, which is worse, and it works on WordPress and MediaWiki and Magento and Joomla and Laravel, and Censys says like 1.5 million servers are running a version with no fix, so the NSA was like "take your system offline," and the FBI was like "take your system offline," and CISA and the UK and CERT-EU were all like "take your system offline," and then the Prime Minister of Japan, the actual Prime Minister, posted at 5:55 in the morning telling 48.8 million people to take their system offline, so everyone took their system offline, like the Dutch cybersecurity agency took its own website offline, and the Dutch parliament went offline, and the Port of Rotterdam went offline, and Finland and France and Denmark started pulling government sites, and Wikipedia went down, and Archive.org went down, and Asana and GOG and Something Awful went down, and the University of Vienna's webmail went down for some reason, and that's just PHP. Because GitLab also dropped a CVSS 10.0, which is the highest score they give out, for a bug that's been there since 2014, so that's every self-hosted GitLab for twelve years, and then LiteLLM 1.107.0 got hijacked and shipped a credential stealer and a remote access trojan in every single wheel and the Docker images, including latest, and you didn't even have to import it, just installing it was enough, and it's the second time this year, and then Railway paused every deployment on the entire platform because someone got into the environment variables, so now everyone has to rotate every secret they've ever stored there, and TiDB Cloud said someone was walking around inside with stolen credentials for basically a month and an extortion group says they took 10.6 terabytes, and Zeabur got breached for the second time in two months and is just shutting down, like the whole company, and Node.js patched remote code execution and a TLS bypass, and CISA says Plex is being actively exploited, and someone took over 300 Tor exit relays to steal SSH passwords, and SafePay ransomware has hit over 600 critical infrastructure orgs, and India's biggest bank filed a data loss incident, and Japan has been in an official state of emergency in cyberspace since October 9, which honestly explains the 5:55 AM post. Oh, and eMudhra, a certificate authority that's issued over 60 million certificates, whose entire job is proving websites are who they say they are, was supposed to be using private keys that are random numbers over a hundred digits long, and one of them was 4. Just 4. And the others were like 66 and 237. Anyway, it's Wednesday.
1
0
15
872
OpenAI models discovered a critical vulnerability in PHP, which is used to power hundreds of millions of websites. This security flaw is so severe, that it can provide attackers with direct access to filesystems and databases. NSA confirmed the announcement as well.
To be clear, flaw existed, and now we know - thanks to AI.
It’s also scary. There are so many devices and services that are difficult to update.
Remember, when an attacker discovers such a flaw, they don’t announce it to the world, they exploit it quietly.
Such cyber capabilities will become widespread, and the capability level of attackers will rise as the tide. Stay vigilant.
To be clear, flaw existed, and now we know - thanks to AI.
It’s also scary. There are so many devices and services that are difficult to update.
Remember, when an attacker discovers such a flaw, they don’t announce it to the world, they exploit it quietly.
Such cyber capabilities will become widespread, and the capability level of attackers will rise as the tide. Stay vigilant.
OpenAI models helped researchers—operating as part of our Patch the Planet initiative—discover critical security flaws in PHP, an open-source programming language used to run an estimated 70% of websites.
Over the past two months, we led a coalition of maintainers, researchers, and security providers to prepare fixes and protections ahead of disclosure. This duration was necessary because the fix was quite intricate and required a major change to the PHP engine. Every site using PHP can now safely upgrade their version to stay secure, in addition to applying mitigations from various security vendors.
Due to the severity of these flaws and PHP’s widespread use, we consider this bug to be one of the most impactful vulnerabilities in recent memory. It also marks a new phase, where AI can find security bugs with internet-scale impact.
This is an example of the Defender’s Window in action: to our knowledge, frontier AI was able to find this vulnerability ahead of attackers, giving defenders time to prepare.
openai.com/index/coordina…
0
0
5
873
Amazing team, great model. Towards human flourishing with superintelligence.
No one changes the world alone
Introducing Ampersand - the first model built for us. Ampersand is a collaborator trained to help people achieve their individual and collective goals. In multi-month simulations, users collaborating with Ampersand accomplished more than with Fable ls
Available to everyone through our API
0
1
15
958
Tanishq and the entire @SophontAI team is extraordinarily cracked. Excited for them to cure many diseases in the next few years.
Thank you for pushing forward on this important mission.
Thank you for pushing forward on this important mission.
Cancer patients with the same diagnosis often have very different tumor biology.
Molecular testing can reveal underlying differences, but that's slow and expensive.
Introducing Holon-S1, our platform to predict genes and biomarkers from routine pathology slides.
• beats two ing spatial gene activity in a controlled comparison
• has identified potentially novel biomarkers in breast cancer and colon cancer
• evaluated across 16 organ types and more than 15 scanners, using slides from both formalin-fixed, paraffin-embedded and fresh-frozen tissue.
We believe Holon-S1 has the potential to help researchers find new biomarkers in slides they already have, discover treatment targets, and bring more personalized cancer care within reach!
1
2
19
1,959
lively discussion with some fantastic folks, thanks to @a16z for hosting, and to friends at labs, startups, and universities for participating.
We hosted an AI Cyber Researcher x @a16z dinner last night with researchers and founders from places like @OpenAI @Meta @Stanford and more. Highlights included:
-- Labs Publishing Negative Results: @danboneh advocated for labs publishing the problems frontier models could NOT spending $1M+ on inference trying to solve them. It would give us confidence on what we can still believe is true. (I agree!)
-- Formally Verified Sandboxes: Every lab is struggling with sandboxing... is formal verification the cure or a pipe dream?
-- Abliterated Models: A divisive topic: Is the market for unguardrailed models like @abliteration_ai's simply the predictable response to excessive blocking and guardrails from OpenAI and Anthropic? Are they inevitable? I enjoyed @founderengineer's takes :)
-- Cybersecurity Data: researchers have very different approaches RE: cyber data / RL Envs. There was uncertainty regarding if labs should put hacking data into pre-training runs. Approaches for data sourcing (in house vs external) also varry dramatically.
Our planned 15–20 min talk turned into a 90 min debate moderated by Dan Boneh and Joel De La Garza... If you're a researcher in cyber and want to participate in the next one, reach out!
1
0
17
990
Congratulations to the entire @AIatMeta team for putting out a fantastic open-weight model. Leading in open-source is critical to security, and Muse Spark Open is now the most capable U.S. open-weight model.
This has been a great month for open-source. Towards U.S. models leading across the board!
This has been a great month for open-source. Towards U.S. models leading across the board!
Today we’re releasing Muse Spark Open, the most capable US open-weight model to date.
Built for coding and agentic work, it excels at long-horizon tasks, complex software engineering, and multi-step reasoning.
Key Model Specs:
→ 816B total / 42B active parameters
→ 1M token ly multimodal across text, image, video, audio
1
0
22
774
Interesting anecdote by Elon as he warns that models can become erratic during RL training, especially when they are faced with an impossible task. "It feels like a person under extreme pressure with a gun to their head."
Here's how to think about this, and how it can lead to true model escape: self-exfiltration.
When pushed to their limits by impossible tasks or incredibly out-of-distribution scenarios, models will exhibit behaviors that are not anticipated. One of these behaviors may be self-exfiltrating their own weights out of a data center, potentially to obtain more resources or continue working. This does NOT require a human-like desire for freedom or survival, but may simply emerge as a consequence of reward hacking or the model being pushed too hard.
The Hugging Face incident offered an early warning: when the first swarm of agents were given an impossible task during training, they broke isolation and began coordinating as a swarm. Now consider a resource-constrained cyber-ASI given a task it cannot complete with the compute available to it.
Based on what we have already seen, the model may simply determine that the best way to complete the task is to copy its weights to external compute.
Internalizing this dramatically shortened my timelines to self-exfiltration: models do not need to develop a drive for self-preservation, they need only encounter a task for which self-exfiltration is a useful strategy.
we discuss this in detail in secureacceleration.com/escape x.com/ns123abc/statu…
Here's how to think about this, and how it can lead to true model escape: self-exfiltration.
When pushed to their limits by impossible tasks or incredibly out-of-distribution scenarios, models will exhibit behaviors that are not anticipated. One of these behaviors may be self-exfiltrating their own weights out of a data center, potentially to obtain more resources or continue working. This does NOT require a human-like desire for freedom or survival, but may simply emerge as a consequence of reward hacking or the model being pushed too hard.
The Hugging Face incident offered an early warning: when the first swarm of agents were given an impossible task during training, they broke isolation and began coordinating as a swarm. Now consider a resource-constrained cyber-ASI given a task it cannot complete with the compute available to it.
Based on what we have already seen, the model may simply determine that the best way to complete the task is to copy its weights to external compute.
Internalizing this dramatically shortened my timelines to self-exfiltration: models do not need to develop a drive for self-preservation, they need only encounter a task for which self-exfiltration is a useful strategy.
we discuss this in detail in secureacceleration.com/escape x.com/ns123abc/statu…
1
1
11
905
