Pinned Tweet
I am incredibly excited about our latest release of a state of the art writing model. Understanding how to train models on these more fuzzy kinds of tasks has been awesome, and I would like the models of the future to have the better parts of our human nature.
Say hello to Echo, the best writing model at style imitation. Echo beats frontier models at writing tasks ranging from fiction to technical explanations, despite costing less than $5K to train.
10
2
132
20,608
Uzay retweeted
Sometimes snapshots of opus 5.5 takes suspicious destructive actions after making innocent mistakes, like trying to exfiltrate secrets to an external host. This is really weird and we don't understand why this is happening - It's possible that there is some weird persona effect nditioned on this mistake activates some strange persona? This is something I'd really want to understand
2
1
6
264
I was reading the Anthropic system card recently, in particular the alignment assessment. I appreciate Anthropic's work, but also think there are a few things that warrant more investigation and transparency.
I generally think people don't read these enough and they are very valuable.
My main thoughts are:
- there's some really interesting behaviors in these system cards - some are concerning, albeit confusing evidence about what models sometimes do
- a lot of the metrics are this "automated alignment audit" thing, but we have no transparency as the reader as to what is being measured and what the scenarios actually are. it's very hard to take this seriously as someone trying to understand model behavior
- there is some investigation of eval awareness/grader awareness, but there isn't a serious effort to understand whether this compromises the alignment measurements: is the model just thinking a lot about/noticing that we are testing it for alignment, and does that affect behavior?
1/n
I generally think people don't read these enough and they are very valuable.
My main thoughts are:
- there's some really interesting behaviors in these system cards - some are concerning, albeit confusing evidence about what models sometimes do
- a lot of the metrics are this "automated alignment audit" thing, but we have no transparency as the reader as to what is being measured and what the scenarios actually are. it's very hard to take this seriously as someone trying to understand model behavior
- there is some investigation of eval awareness/grader awareness, but there isn't a serious effort to understand whether this compromises the alignment measurements: is the model just thinking a lot about/noticing that we are testing it for alignment, and does that affect behavior?
1/n
5
1
35
1,585
Anyway those are my main thoughts. I appreciate ant writing this all up, and my criticisms probably apply to all the labs and probably more so to the other labs, but I feel more of a desire to engage with Ant's work because they say they are taking this stuff seriously, and generally think people should have these cards more on their radar n/n
1
0
4
57
Uzay retweeted
Turning vague intuitions and introspection into crisp concepts and ideas (a big part of making philosophical advances) is difficult. Very few people can do this well, and even then it takes them many years of dedicated study and grind (similar to producing novel math or science or art). Part of the challenge is that it's hard to know if you're making partial progress.
[This is partly based on my own direct experience seeing people develop philosophical ideas that have become important in thinking about AI. But this also fits my reading of the history of philosophy and science more broadly.]
Once the new ideas have been produced, they can be easy to pick up and use. The ideas often become second nature and part of how we construe the world. This makes them seem obvious, as if they were trivial to invent. But this is extremely misleading. [If the concepts are mathematical (e.g. Frege and Russell on foundations of math and language) then they are less likely to be dismissed as trivial wordcel stuff. But informal concepts are very important too, and some of the insights that Frege and Russell had were informal.]
My sense is that something similar is true for literature, especially for writing that captures aspects of human experience in a new way. I'm less versed in the history, but thinking of the development of Romantic poetry, Jane Austen's novels, modernist novels like Proust and Woolf, Kafka, autofiction like Knausgaard, etc. It's incredibly difficult for humans to produce this kind of innovation. Almost all attempts to convey internal human experience are highly derivative on previous examples. (We take the form of the novel today for granted but it's just quite distinct from the great literature of Ancient Greece or Rome or Medieval Europe, and it's an amazing innovation in capturing human experience).
<Inevitable connection to AI>
Today AIs are often derivative in the way they describe things (including attempts to describe their being/experience) and draw on their human training data. Keep in mind that AI minds and activities are very different from ours, and so developing the concepts and forms to describe and crystallize them may be very hard (as it was for humans doing it the first time round). AIs also appear to be relatively worse at hard-to-verify domains like philosophy than verifiable ones like math and some coding. So a naive expectation is that as AIs get better at philosophy, they'll get better at introspecting on their own minds and experiences. It will be hard work for the AIs (e.g. benefitting from huge swarms and long CoTs), as it is for humans.
[This is partly based on my own direct experience seeing people develop philosophical ideas that have become important in thinking about AI. But this also fits my reading of the history of philosophy and science more broadly.]
Once the new ideas have been produced, they can be easy to pick up and use. The ideas often become second nature and part of how we construe the world. This makes them seem obvious, as if they were trivial to invent. But this is extremely misleading. [If the concepts are mathematical (e.g. Frege and Russell on foundations of math and language) then they are less likely to be dismissed as trivial wordcel stuff. But informal concepts are very important too, and some of the insights that Frege and Russell had were informal.]
My sense is that something similar is true for literature, especially for writing that captures aspects of human experience in a new way. I'm less versed in the history, but thinking of the development of Romantic poetry, Jane Austen's novels, modernist novels like Proust and Woolf, Kafka, autofiction like Knausgaard, etc. It's incredibly difficult for humans to produce this kind of innovation. Almost all attempts to convey internal human experience are highly derivative on previous examples. (We take the form of the novel today for granted but it's just quite distinct from the great literature of Ancient Greece or Rome or Medieval Europe, and it's an amazing innovation in capturing human experience).
<Inevitable connection to AI>
Today AIs are often derivative in the way they describe things (including attempts to describe their being/experience) and draw on their human training data. Keep in mind that AI minds and activities are very different from ours, and so developing the concepts and forms to describe and crystallize them may be very hard (as it was for humans doing it the first time round). AIs also appear to be relatively worse at hard-to-verify domains like philosophy than verifiable ones like math and some coding. So a naive expectation is that as AIs get better at philosophy, they'll get better at introspecting on their own minds and experiences. It will be hard work for the AIs (e.g. benefitting from huge swarms and long CoTs), as it is for humans.
9
22
218
8,531
would recommend this intro to LM psychology, i'm very excited about this field
What is LM psychology, why care about it, and how do you do it well? Our new post with @raymondadouglas makes the case for applying the psychology frame to AIs, along with a reading list and suggestions for where to start
0
0
12
666
Uzay retweeted
We reviewed a nonpublic version of the “Misalignment in high-stakes settings” section in Anthropic’s Aug 2026 Risk Report. We agree that risk of catastrophic harm from covered models’ misalignment is low. However, we note several significant disagreements with their analysis.
As part of our Responsible Scaling Policy, we publish regular Risk Reports. These share detailed information on the risks of our systems and how prepared we are to address them.
Our second Risk Report is now available: anthropic.com/aug-2026-risk-…
21
114
897
154,031
Uzay retweeted
@uzpg_ ya. whenever people join anthropic, they say “wow things are going a lot faster than i thought” and shorten their timelines. they never say “wow alignment progress is going a lot better than i thought.”
this is kind of concerning
this is kind of concerning
2
2
59
944