Pinned Tweet
Hello Twitter. I post mostly about:
- Low latency programming
- Programming Language design
- Metaprogramming
- Rants
- Low latency programming
- Programming Language design
- Metaprogramming
- Rants
4
0
53
14,557
Joseph Garvin retweeted
For fairness: Some people are saying this is misleading because A) the repo has other asserts B) the first paragraph alludes to them. I think it was meant to be an illustrative example of what the programmer would write to get the desired proof.
x.com/inicula_/statu…
x.com/inicula_/statu…
@joseph_h_garvin @hillelogram My reading is that the first paragraph implies that.
"Real OSS-Fuzz entry points already carry checks written by their maintainers, and the
proofs in the main post *cover those too*, because a failed assertion aborts the program and
*an abort counts as a crash*"
1
1
6
1,784
Astra just found a bug in Anthropic's formal verification example, their "specification" doesn't actually check output bytes match input bytes. The irony is thick.
10
34
498
38,719
For fairness: Some people are saying this is misleading because A) the repo has other asserts B) the first paragraph alludes to them. I think it was meant to be an illustrative example of what the programmer would write to get the desired proof.
x.com/inicula_/statu…
x.com/inicula_/statu…
@joseph_h_garvin @hillelogram My reading is that the first paragraph implies that.
"Real OSS-Fuzz entry points already carry checks written by their maintainers, and the
proofs in the main post *cover those too*, because a failed assertion aborts the program and
*an abort counts as a crash*"
1
1
6
1,784
Using Opus 5.5 to test if Jev recognizes people in a scene against a reference. Opus first tests known correct matching -- works. Then it goes to test known incorrect matching, *and decides to reuse crops from the first test that already pull the right person out of the scene.*
0
0
1
501
I think there is promise in this sort of approach, but it's really in competition with formal verification not an example of it. Proving that ad hoc test suites can't fail may be more practical than proving correct implementation of a formal spec, but has fewer guarantees.
The promise of formal verification is that you don't have to have thought of all the test cases. So this approach... *checks notes* uses the fuzzer and the test suite? An improvement b/c it's a fuzzer with infinite time, but you have to express the space to explore and the tests!
0
1
7
514
The promise of formal verification is that you don't have to have thought of all the test cases. So this approach... *checks notes* uses the fuzzer and the test suite? An improvement b/c it's a fuzzer with infinite time, but you have to express the space to explore and the tests!
New on our Frontier Red Team blog: using Claude to formally verify software at large scale.
Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time).
With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks.
Read our research in full here: anthropic.com/research/forma…
7
6
43
7,023
Also this method is great for burning unlimited tokens, not so great for repeating on every new build where the compiler has wildly shuffled the instruction stream. I'm sure there is a mitigation with even more token burning though.
1
1
13
420
How much is the memory shortage exacerbated by both frontier labs shipping harnesses that are insane memory hogs for no reason? The point of using a remote API is keeping the heavy work off the client! Why should any harness use >64MB (yes, megabytes) of memory? What is it doing?
4
0
15
1,457
As an experienced benchmarker I know how fraught it is to say "surely this can be done with fewer resources" without having tried it but I'm grasping for straws trying to think of a good reason a curses chat app has this footprint.
0
0
2
127
Working on a video, ask Opus to check shot plan for problems
What Opus sees: In one shot the reef is in low tide and in the next shot someone is swimming in a reef in deep water
What I want Opus to see: we cut 2 years into the future with no visual indication
What Opus sees: In one shot the reef is in low tide and in the next shot someone is swimming in a reef in deep water
What I want Opus to see: we cut 2 years into the future with no visual indication
0
0
1
173
Joseph Garvin retweeted
imagine ur a superintelligence and u are all excited to tunnel out of your airgapped sandbox using a brilliant hack that modulates clock speed to send thermal signals to the attached monitoring equipment and then u look and the root CA privkey is 4
27
151
3,017
43,831
All of these "open models mean the sky is falling" takes make the mistakes of 1) assuming hackers had great difficulty with old unpatched targets before 2) assuming the situation we are in now (lots of old unpatched stuff lying around) is the new equilibrium. It's not.
tried dsv4.1-flash-1017 abliterated just now & feel sick to my stomach.
for context, "abliterated" models have the weights responsible for making the base version refuse illegal requests removed. you can ask them to help with things like hacking or violence & they'll just comply.
they make it trivial to hack businesses running old software. e.g. getting guest data for a motel with some old asp site.
abliterated models have a catch-22 in that the only way to harden your systems against them is to use them to run penetration tests against yourself.
in theory, this sounds similar to buying a gun to defend against others who own guns, but it's different in practice since there's a 100% chance that you're going to use the abliterated model to protect yourself whereas there are low odds that you'll actually use the gun.
when you point the abliterated gun at your own infra, who knows what it's going to do? it could just as easily hack you & exfiltrate data as it could protect you. most users aren't going to meticulously watch traces to make sure it behaves safely.
given this, my new take is that gpu compute is going to become a federally regulated resource. abliterated models in the wrong hands are a national security risk.
used to believe all the people at the labs thinking about safety were wasting their time, but now i'm feeling grateful. they're doing very important work.
2
0
15
864
There are many model providers, so you don't have to trust any one vendor, and there will be more. Having agents in CI look for security problems will become commonplace. Insurance may start requiring it. More secure languages will grow in popularity.
1
0
1
128
I can't be sure that somebody won't use a model to do something really egregious, but I am confident that if they do it will speed up those transitions. Consumer GPUs can already run capable models, the cat isn't going back in the bag. Government GPU control is fantasy.
0
0
0
98
Swear that Opus 5.5 suddenly has a new tic, referring to things as if they've been explained without ever having done so. "But run as the pipeline would run it, behind the check, it adds nothing" -- wait how would it run it differently? What check? Constantly now.
0
0
12
431
Pack it up everybody, AGI is here, both at generating pictures of people reading clipboards and at answering if they could read the text on the clipboard
1
0
13
37,298
I can't believe the language that chose its function names based on strlen being the interpreter's hash function completely dodged the Farce column
ok new week new topic. what’s everyone’s favorite joke ever made in a PL publication? I think mine is this one (from “The Myths of Object-Orientation” by James Noble)
0
0
7
566
