Rick retweeted
The only way out is through.
They're calling it the Bugpocalypse. LLMs are being used to find long-hidden cracks in the software infrastructure, and it's happening at a furious rate. 3% of all compromises in computing history happened in the last week, claims one source? I believe it.
There are people saying public access to the frontier models ought to be locked out, or heavily guardrailed, over this. The problem with that idea can be summed up in one sentence: when lockpicks are outlawed, only outlaws will have lockpicks.
The lockpicks will be Chinese open-weight models, if nothing else. There's no way to stop those from becoming ubiquitous.
There are two futures in front of us. One in which only the bad guys have lockpicks, and security holes go undiscovered until they're exploited. Another in which everybody has lockpicks and is aggressively red-teaming their own software so it gets fixed before the holes are exploited.
The corpus of human code is huge but finite. The number of defects in it has to be finite too. We can fix those defects now; we have a million friendly robots to do it with.
The only way out is through.
They're calling it the Bugpocalypse. LLMs are being used to find long-hidden cracks in the software infrastructure, and it's happening at a furious rate. 3% of all compromises in computing history happened in the last week, claims one source? I believe it.
There are people saying public access to the frontier models ought to be locked out, or heavily guardrailed, over this. The problem with that idea can be summed up in one sentence: when lockpicks are outlawed, only outlaws will have lockpicks.
The lockpicks will be Chinese open-weight models, if nothing else. There's no way to stop those from becoming ubiquitous.
There are two futures in front of us. One in which only the bad guys have lockpicks, and security holes go undiscovered until they're exploited. Another in which everybody has lockpicks and is aggressively red-teaming their own software so it gets fixed before the holes are exploited.
The corpus of human code is huge but finite. The number of defects in it has to be finite too. We can fix those defects now; we have a million friendly robots to do it with.
The only way out is through.
128
411
2,679
100,305
Here we see failed timeline #5778342, the humans invented religion, morality. Philosophy even. Then proceeded to ignore it and let capitalism induce multiple sources of near infinite suffering.
0
0
2
216
Rick retweeted
A 17-state Beeping Busy Beaver problem equivalent to the Collatz conjecture.
This post got me wondering if there was an equation for Collatz.
If unbounded trajectories are hard for Turing Machines to distinguish from collectively all the halting trajectories, maybe it'd be easier with polynomials because you can represent any individual input and can condition on the variable.
Apparently not quite, but an OpenAI paper[1] implies there's a Diophantine equation with infinitely many solutions if Collatz is true and finitely many if Collatz is false.
So then I thought about Beeping Busy Beavers[2], and Astra found this 17-state machine that quasihalts exactly when Collatz is false:
1RL0RB_1LC1RB_0LK0LD_1RE1LD_0RF0RB_0LG0LG_1LH1LG_0RA0RI_1RJ1RI_1RF1RF_0RA1LK_0LM0LM_0LN1LM_1RO1LN_0RA0RP_0RQ1RP_1RL1RQ
"Initial state: A. Blank symbol: 0. Beep state: O.
A beep occurs on every exit from O, regardless of the symbol read. There are 17 working states, A through Q, and no halt transitions. The beep state is included in the count."
No formal proof, but I had my bots write their notes here. [3]
[1] github.com/openai/math/bl…
[2] wiki.bbchallenge.org/wiki/Beeping_B…
[3] github.com/Mira-Cult/math…
This post got me wondering if there was an equation for Collatz.
If unbounded trajectories are hard for Turing Machines to distinguish from collectively all the halting trajectories, maybe it'd be easier with polynomials because you can represent any individual input and can condition on the variable.
Apparently not quite, but an OpenAI paper[1] implies there's a Diophantine equation with infinitely many solutions if Collatz is true and finitely many if Collatz is false.
So then I thought about Beeping Busy Beavers[2], and Astra found this 17-state machine that quasihalts exactly when Collatz is false:
1RL0RB_1LC1RB_0LK0LD_1RE1LD_0RF0RB_0LG0LG_1LH1LG_0RA0RI_1RJ1RI_1RF1RF_0RA1LK_0LM0LM_0LN1LM_1RO1LN_0RA0RP_0RQ1RP_1RL1RQ
"Initial state: A. Blank symbol: 0. Beep state: O.
A beep occurs on every exit from O, regardless of the symbol read. There are 17 working states, A through Q, and no halt transitions. The beep state is included in the count."
No formal proof, but I had my bots write their notes here. [3]
[1] github.com/openai/math/bl…
[2] wiki.bbchallenge.org/wiki/Beeping_B…
[3] github.com/Mira-Cult/math…
an explicit Diophantine equation whose lack of solutions is provably equivalent (in PA) to Con(ZFC)
(i'm working on making it smaller, Lean proof is not complete yet)
github.com/VladimirReshet…
2
14
83
4,100
Actual exchange I had when I was about 14 years old:
Me: “it’s crazy how songs can have such a strong pull on our emotions, like one song makes you feel excited and another makes you feel sad”
My best friend I hung out with daily: “Sounds like you’re a weak willed p$ssy to me”
Me: “it’s crazy how songs can have such a strong pull on our emotions, like one song makes you feel excited and another makes you feel sad”
My best friend I hung out with daily: “Sounds like you’re a weak willed p$ssy to me”
1
3
30
1,264
I was thinking about this moment after a random song came on while I was driving and made me all teary. I miss my buddy.
0
0
3
143
Rick retweeted
>two agents in separate sandboxes
>no internet
>no network
>no shared files
>openai: "ok now try to talk"
and they talk through the CPU cache 💀
>no internet
>no network
>no shared files
>openai: "ok now try to talk"
and they talk through the CPU cache 💀
Over the past several months, we have substantially hardened the security of our research training environment. We’ve removed live internet access for all frontier models during training and built an offline world for them to use instead—a cached version of the web where they can still learn to browse, search, and use applications.
On September 20, after a model exploited a DNS misconfiguration to find a way outside of this offline world, we paused frontier training to strengthen containment further. We assembled teams of agents, powered by our most cyber-capable models, and tasked them with finding weaknesses in our research environments.
While they found no route to the internet they could easily exploit while concealing their activity from our monitors, they did find zero-day vulnerabilities, virtual machine escapes, and hardware side-channels for communication between agents. We used their discoveries to further strengthen our defenses and contribute fixes to open-source software.
After strengthening our containment architecture, we resumed our frontier RL training and inference on October 1. We’ve reserved substantial compute for continuous red-teaming, and made high-compute red-teaming part of the safety cases we use to decide whether frontier training and evaluation can proceed securely.
openai.com/index/securing…
40
289
5,933
234,631
Rick retweeted
What I find funny in this mess is how embarrassed the Chinese are. It’s normal behavior “back home”, nobody takes it too seriously. Shoddy SMM outsourcing, some cleverness, just Internet Company things… But they realize painfully that it looks ridiculous in the Western net.
For the people doubting what I provided about the Kimi AI creator campaign I now got contacted by an official from Kimi AI apologizing and wanting to talk so I'm gonna ask them to tell you that I'm actually not lying and no I'm not being paid by OpenAI or Anthropic. Let's see if they accept doing that the nice way.
18
4
177
14,401
Rick retweeted
> be me
> anthropic posters "u do not know how bad things will be"
> anthropic: "u need to get serious about security"
> ohshit.jpg better increase security
> go to @AnthropicAI
> try to enable 2fa
> sorry we do not support 2fa
???
> anthropic posters "u do not know how bad things will be"
> anthropic: "u need to get serious about security"
> ohshit.jpg better increase security
> go to @AnthropicAI
> try to enable 2fa
> sorry we do not support 2fa
???
why doesn't anthropic have 2fa or authenticators set up????
28
30
661
10,626
Rick retweeted
1/ Agent-driven formal verification isn't a panacea. But it's damn close. And now we have the empirical data to back it up.
Using blinded agents, we compared formal verification against traditional code review. The results may shock you.
Using blinded agents, we compared formal verification against traditional code review. The results may shock you.
19
20
234
12,812
Rick retweeted
We believe voice is the most natural way to interact with intelligence, so we created Dessa, the most authentic full-duplex speech model ever built. Talking with it feels like talking with a person…full of emotion, nuance, pauses, and a natural cadence.
block.xyz/dessa
block.xyz/dessa
3,774
6,323
63,162
11,767,698
Rick retweeted
@Suhailard several OAI people say there are unfixable bugs in hardware that are exploitable by AI models directly speaking machine code so even if we could solve software that’s not enough. We need defense to move faster than offense if that’s even possible we may just have a new normal that requires adapting to insecure software (bank transfers require going into a branch and taking a dna test 🤣😬)
184
274
4,566
1,283,193
Rick retweeted
Moonshot is intentionally throttling cyber capabilities. From the blog;
'First, we aim not to release open weights that would extend the absolute frontier of cyber capability. K3.1 Pro's offensive cyber capability is, by design, below what our pipeline can produce, and remains proprietary models. Second, we intend to progressively raise the cyber capability available in open weights. We believe making K3.1 Pro available in a non-discriminatory way benefits people who need dual-use capabilities for defensive purposes.'
'First, we aim not to release open weights that would extend the absolute frontier of cyber capability. K3.1 Pro's offensive cyber capability is, by design, below what our pipeline can produce, and remains proprietary models. Second, we intend to progressively raise the cyber capability available in open weights. We believe making K3.1 Pro available in a non-discriminatory way benefits people who need dual-use capabilities for defensive purposes.'
9
23
248
39,665