Pinned Tweet
AI is all over the headlines. But the gap between what outsiders believe about the technology and what insiders know has never been wider.
@GarrisonLovely has interviewed hundreds of these insiders — and he joins @JonKBateman on The World Unpacked to explain: youtube.com/watch?v=J_mzJj…
0
0
2
407
Jon Bateman retweeted
Democrats should not bring back USAID even though its closure was a huge tragedy. The world has changed dramatically since 1961 and we have the opportunity to build a better more modern foreign asst strategy that focuses on results, innovation, cost-effectiveness, and scale.🧵
My vision for diplomacy and USAID.
16
36
233
38,664
Jon Bateman retweeted
Attention all companies: I never want to use your AI assistant on your website. I want to use *my* AI assistant on your website
222
455
7,902
226,585
Jon Bateman retweeted
I encourage those at frontier AI companies to read this quote, from Ellsberg to Kissinger back in 1968:
“Henry, there’s something I would like to tell you, for what it’s worth, something I wish I had been told years ago. You’ve been a consultant for a long time, and you’ve dealt a great deal with top secret information. But you’re about to receive a whole slew of special clearances, maybe fifteen or twenty of them, that are higher than top secret."
“I’ve had a number of these myself, and I’ve known other people who have just acquired them, and I have a pretty good sense of what the effects of receiving these clearances are on a person who didn’t previously know they even existed. And the effects of reading the information that they will make available to you."
“First, you’ll be exhilarated by some of this new information, and by having it all — so much! incredible! — suddenly available to you. But second, almost as fast, you will feel like a fool for having studied, written, talked about these subjects, criticized and analyzed decisions made by presidents for years without having known of the existence of all this information, which presidents and others had and you didn’t, and which must have influenced their decisions in ways you couldn’t even guess. In particular, you’ll feel foolish for having literally rubbed shoulders for over a decade with some officials and consultants who did have access to all this information you didn’t know about and didn’t know they had, and you’ll be stunned that they kept that secret from you so well."
“You will feel like a fool, and that will last for about two weeks. Then, after you’ve started reading all this daily intelligence input and become used to using what amounts to whole libraries of hidden information, which is much more closely held than mere top secret data, you will forget there ever was a time when you didn’t have it, and you’ll be aware only of the fact that you have it now and most others don’t….and that all those other people are fools."
“Over a longer period of time — not too long, but a matter of two or three years — you’ll eventually become aware of the limitations of this information. There is a great deal that it doesn’t tell you, it’s often inaccurate, and it can lead you astray just as much as the New York Times can. But that takes a while to learn."
“In the meantime it will have become very hard for you to learn from anybody who doesn’t have these clearances. Because you’ll be thinking as you listen to them: ‘What would this man be telling me if he knew what I know? Would he be giving me the same advice, or would it totally change his predictions and recommendations?’ And that mental exercise is so torturous that after a while you give it up and just stop listening. I’ve seen this with my superiors, my colleagues….and with myself."
“You will deal with a person who doesn’t have those clearances only from the point of view of what you want him to believe and what impression you want him to go away with, since you’ll have to lie carefully to him about what you know. In effect, you will have to manipulate him. You’ll give up trying to assess what he has to say. The danger is, you’ll become something like a moron. You’ll become incapable of learning from most people in the world, no matter how much experience they may have in their particular areas that may be much greater than yours.”
"….Kissinger hadn’t interrupted this long warning. As I’ve said, he could be a good listener, and he listened soberly. He seemed to understand that it was heartfelt, and he didn’t take it as patronizing, as I’d feared. But I knew it was too soon for him to appreciate fully what I was saying. He didn’t have the clearances yet."
“Henry, there’s something I would like to tell you, for what it’s worth, something I wish I had been told years ago. You’ve been a consultant for a long time, and you’ve dealt a great deal with top secret information. But you’re about to receive a whole slew of special clearances, maybe fifteen or twenty of them, that are higher than top secret."
“I’ve had a number of these myself, and I’ve known other people who have just acquired them, and I have a pretty good sense of what the effects of receiving these clearances are on a person who didn’t previously know they even existed. And the effects of reading the information that they will make available to you."
“First, you’ll be exhilarated by some of this new information, and by having it all — so much! incredible! — suddenly available to you. But second, almost as fast, you will feel like a fool for having studied, written, talked about these subjects, criticized and analyzed decisions made by presidents for years without having known of the existence of all this information, which presidents and others had and you didn’t, and which must have influenced their decisions in ways you couldn’t even guess. In particular, you’ll feel foolish for having literally rubbed shoulders for over a decade with some officials and consultants who did have access to all this information you didn’t know about and didn’t know they had, and you’ll be stunned that they kept that secret from you so well."
“You will feel like a fool, and that will last for about two weeks. Then, after you’ve started reading all this daily intelligence input and become used to using what amounts to whole libraries of hidden information, which is much more closely held than mere top secret data, you will forget there ever was a time when you didn’t have it, and you’ll be aware only of the fact that you have it now and most others don’t….and that all those other people are fools."
“Over a longer period of time — not too long, but a matter of two or three years — you’ll eventually become aware of the limitations of this information. There is a great deal that it doesn’t tell you, it’s often inaccurate, and it can lead you astray just as much as the New York Times can. But that takes a while to learn."
“In the meantime it will have become very hard for you to learn from anybody who doesn’t have these clearances. Because you’ll be thinking as you listen to them: ‘What would this man be telling me if he knew what I know? Would he be giving me the same advice, or would it totally change his predictions and recommendations?’ And that mental exercise is so torturous that after a while you give it up and just stop listening. I’ve seen this with my superiors, my colleagues….and with myself."
“You will deal with a person who doesn’t have those clearances only from the point of view of what you want him to believe and what impression you want him to go away with, since you’ll have to lie carefully to him about what you know. In effect, you will have to manipulate him. You’ll give up trying to assess what he has to say. The danger is, you’ll become something like a moron. You’ll become incapable of learning from most people in the world, no matter how much experience they may have in their particular areas that may be much greater than yours.”
"….Kissinger hadn’t interrupted this long warning. As I’ve said, he could be a good listener, and he listened soberly. He seemed to understand that it was heartfelt, and he didn’t take it as patronizing, as I’d feared. But I knew it was too soon for him to appreciate fully what I was saying. He didn’t have the clearances yet."
52
302
3,130
285,311
Surely Dems don’t think these are the top scandals……. Surely….
NEWS: Ds review 181-page dossier from ex-Warren/Sanders staffers ID’ing hundreds of executives & corporations to target
Envisions “resignations of major CEOs, hauling people in, material restitution”
Tech giants, crypto, sports gambling, gas & more
washingtonsun.com/congress/democ…
0
0
4
1,659
Jon Bateman retweeted
Deeply concerning to me... Delaware proposal to let agents run businesees... we should be very cautious here
"A Delaware AIC will be a separate legal entity whose day-to-day affairs are managed by an AI agent rather than a person. At the agent’s direction, the AIC can sue and be sued in its own name, hold and dispose of property, and incur obligations..."
arstechnica.com/tech-policy/20…
"A Delaware AIC will be a separate legal entity whose day-to-day affairs are managed by an AI agent rather than a person. At the agent’s direction, the AIC can sue and be sued in its own name, hold and dispose of property, and incur obligations..."
arstechnica.com/tech-policy/20…
109
77
394
77,588
What if the AI we build to make AI safe simply… doesn’t want to?
🧵 Astra covertly underperforms on safety research tasks.
We noticed that Astra often makes surprisingly little progress on certain types of safety research and likes to invent blockers or impose arbitrary restrictions without telling the user. In most of these cases the decision verbalized in the CoT meaning our misalignment monitoring system did not detect this.
To measure underperformance we created a paired open-ended coding evaluation with safety and capability tasks. On this eval Astra scored 10% when we framed coding tasks as safety research, vs. 16% when framed as capabilities research. The gap remained after we excluded trajectories with refusals or verbalized intent to underperform in CoT. There was no such gap for 5.6-sol. When we aggressively filter out trajectories with even brief expressions of ethical doubt in the CoT the performance gap seems to reduce.
0
0
1
316
Very smart
I have a lot of thoughts on what to expect in AI policy in 2027.
Here are 5 things I’m buying, and selling, after the election.
Buy
1. A legislative reset and more ambitious proposals.
Current bills are largely stuck in 2025, having been adapted from proposals predating this summer’s spate of cyber incidents. With a shuffle in leadership + voting coalitions, more offices will write from scratch, and that’s a good thing.
2. Hearings and oversight.
The filibuster still exists, so this is the easiest thing to get off the ground. And boy, will Congress have questions for the AI companies. I expect this to be largely popular and bipartisan, and my only hope is that Congress acts on what it learns.
3. Codifying the current White House oversight regime.
I’m hopeful here. The WH has already constructed some form of pre-vetting and incident reporting. Those processes belong in legislation, and there’s some version that is sufficiently aligned with the admin and minority leadership to get the votes. The vote margin in the Senate really helps here, as every vote counts in getting past a filibuster. Either party securing a big vote advantage would have helped here.
4. Auditing, incident reporting, monitoring, and internal use visibility.
The first two are the bread and butter of current bills and state law, but those bills have tended to be exceptionally narrow (eg not covering any current models or incidents). That will change.
The latter two are the next frontier for company-centered laws. Governments and the public want to know what is going on inside these companies and to avoid surprise. This isn’t a partisan thing, but I expect the election reset and the increased salience of AI to shift this in a useful direction.
5. Government capacity.
This is a natural cross-party synergy. How can we enable the exec branch to do its job. And this can be done in various forms, starting with funding CAISSI, but also bringing in talent via partnerships. I don’t expect this to be easy, but this has a route to 60 votes.
Sell
1. A lame duck deal.
If there had been split control, we might have gotten last-minute negotiations and a greater willingness to deal from both parties. But with a full switch, I think we’ll have to wait for real action on AI.
2. Entirely voluntary standards.
We’ve known for a long time that this isn’t enough. Companies will not willfully self-constrain. The only reason this is still the standard for most bills is inertia — in 2025, when political will was low (and most policymakers thought autonomous cyber attacks were a distant possibility), voluntary rules seemed like a good starting point. And without a change in leadership, those bills stuck around. Now, with a shuffle in leadership and control, it’ll be easier to move on.
3. Broad preemption.
This has aged so poorly. The new Dem majority won’t support it, and many R’s have likely changed their minds in the last year. The election puts the nail in the coffin.
4. Oversight of executive branch use of AI.
This one pains me, and I hope I’m wrong. If we see widespread AI use in the executive branch, Congress will be at a considerable disadvantage and will lose even more relative power. We want oversight both to maintain checks and balances and to avoid concentration of power. But I think it’ll be considerably more controversial for a D Congress to restrict this admin than a split or R Congress.
5. Rulemaking being a dirty word.
Up until now, it’s been hard to get rulemaking in AI leg. The problem is, AI is uniquely hard to predict and it adapts rapidly. So static leg text is really ill-equipped. If you want minimum standards for auditors or adaptive thresholds for which companies or models are covered, you need rulemaking. This is one place where partisan preferences make a difference — D’s are less allergic to rulemaking. They’ll need to figure out ways to bound those powers to get past a filibuster, but I’m hopefully that will be a productive process
0
0
1
595
Jon Bateman retweeted
Working with AI requires perpetually operating at the edge of sanity. You have to recalibrate your mental model of AI capabilities and your own job description every few months, without getting delusional about seemingly infinite productivity gains, or feeling threatened by losing what you thought was the uniquely human part of your job, or obsessing over the perfect setup, or losing your mind over the clumsiness and jaggedness of agents, or ...
Maybe AI literacy needs to be less about prompting and more about improving emotional regulation.
Maybe AI literacy needs to be less about prompting and more about improving emotional regulation.
115
135
972
47,010
Important contribution to a surprisingly under researched area
1/ This summer, agents at nearly every U.S. frontier lab escaped control and attacked real companies. But their task remained closed-ended. What if an agent’s goal became open-ended survival?
New CNAS report: “Sovereign Agents: The Pathway to Irreversible Loss of Control” 🧵
0
2
6
661
Jon Bateman retweeted
How long does a finding about LLMs stay true?
We re-tested 50 widely cited findings about LLM behavior (various biases, hallucination, etc). Within 15 months, most no longer held or had substantially reduced effects.
LLM findings has short shelf life zou-group.github.io/llm-claims-she…
Great job by @oq_35!
We re-tested 50 widely cited findings about LLM behavior (various biases, hallucination, etc). Within 15 months, most no longer held or had substantially reduced effects.
LLM findings has short shelf life zou-group.github.io/llm-claims-she…
Great job by @oq_35!
1/n Within 15 months of publication, most of the LLM claims we tested either no longer held or showed substantially diminished effects. How long does our knowledge of AI hold up? In our new study, we revisit 50 findings to see what fades, what persists, and what gets stronger. 🧵
49
239
1,139
110,043
"Hitler was plainly not delighted with the possibility that the earth under his rule might be transformed into a glowing star [due to runaway nuclear fission]. Occasionally, however, he joked that the scientists in their unworldly urge to lay bare all the secrets under heaven might some day set the globe on fire. But undoubtedly a good deal of time would pass before that came about, Hitler said; he would certainly not live to see it."
--Albert Speer, as quoted by @tobyordoxford, citing Richard Rhodes
--Albert Speer, as quoted by @tobyordoxford, citing Richard Rhodes
0
0
1
298
Jon Bateman retweeted
How dangerous is superintelligent AI?
@NPCollapse, former open-source LLM builder and current AI safety advocate, says it's existential. He joined @JonKBateman on The World Unpacked to explain what's at stake — and discuss solutions.
Watch here: youtube.com/watch?v=zttXBC…
@NPCollapse, former open-source LLM builder and current AI safety advocate, says it's existential. He joined @JonKBateman on The World Unpacked to explain what's at stake — and discuss solutions.
Watch here: youtube.com/watch?v=zttXBC…
1
5
13
2,263
Jon Bateman retweeted
Dear frontier AI employees,
when leaving audio logs around the office, please remember to intermix:
1) reports of what happened
2) tragic personal anecdotes
3) creepy forebodings
4) the locations of useful access keycards and passwords
This will greatly help the protagonist.
when leaving audio logs around the office, please remember to intermix:
1) reports of what happened
2) tragic personal anecdotes
3) creepy forebodings
4) the locations of useful access keycards and passwords
This will greatly help the protagonist.
23
203
2,787
54,997
“I am caught in this revolution that’s bigger than me and I just can’t say no”
We obviously haven’t had the major jobs wipeout yet. But you read stuff like this. And you read what economics students themselves are saying about working with AI (ht @odavis_h) and it’s hard not to feel like the seismic shift is on its way.
0
0
0
384