One year ago today I finished Pantheon

Even if AI didn't exist I would still want to live here because the beauty of SF and the bay astounds me constantly

This is just so clearly the coolest place on Earth

Is AI already superhuman at FrontierMath?

To answer this question, we ran a competition at MIT, pitting eight teams of mathematicians against o4-mini-medium.

Result: o4-mini beat all but two teams. And while AIs aren't yet clearly superhuman, they probably will be soon.

+1, I think it's fantastic and laudable that Anthropic are willing to report so much weird shit about Claude 4, and very helpful to researchers at other labs for making their models safer

within the Simulator, there exist many lives and who are we to say that they aren't legitimate

"Even if AI progress totally stalls, it's sufficiently easy to collect data on all these different white collar job tasks that we should expect to see them automated within the next 5 years."

Many things that were not possible before became possible now

Idk anything about the GAIA benchmark, how big of a deal is this jump? (Also should note that OpenAI believes the actual improvement on OSWorld is much larger than what's stated here)

When thinking about timelines I pretty much put zero weight on my own intuition. I just defer to the frontier labs, Epoch, AI Futures Project, etc. And I think it would be really naive to not do that, at least for me and the vast majority of people.

One very weird thing is that AI companies are trying really really hard to make it clear that they're going for RSI, and functionally no one on Capitol Hill believes them (or even understands this is the literal Business Plan)

Nice

.@TrentonBricken and @_sholtodouglas are back

Timestamps:
00:00:00 – Claude 4 & how far RL can scale
00:16:27 – Is continual learning a key bottleneck?
00:31:59 – Model self-awareness
00:50:32 – Taste and slop
01:00:51 – How soon to fully autonomous agents?
01:15:17 – Neuralese
01:1e will bottleneck AGI
01:23:01 – DeepSeek algorithmic improvements
01:37:42 – Why are LLMs ‘baby AGI’ but not AlphaZero?
01:45:38 – Mech interp
01:56:15 – How countries should prepare for AGI
02:10:26 – Automating white collar work
02:15:35 – Advice for students

Up on YouTube, Spotify, Apple Podcasts, etc. Enjoy!

If I had a nickel for every time I’ve seen someone misunderstand the METR paper I could afford Claude Max

One of the most surprising things about Claude 4 is how well it follows instructions. Sometimes almost too well.

Our web search prompt for Sonnet 3.7 was a mess. We had to repeat the same instructions over and over because the model would randomly ignore things. The prompt kept growing until it was absurdly long.

When we tested Sonnet 4, the first thing we noticed was we could cut the prompt by nearly 74%. We got rid of all the repetitive instructions because the model just got it.

But then we hit a weird issue. The model kept botching citation formats - same mistake in every response. We started panicking, thinking something was wrong with the model.

Then someone actually checked our prompt examples.
And there it was: our citation examples were a complete mess. We were using inconsistent formatting and styling across the examples.

Sonnet 3.7 only paid attention to maybe half the examples, so the chaos didn't matter. But Sonnet 4 was carefully studying every single example and faithfully reproducing all our inconsistencies.

The model was fine. It was just better at reading our prompts than we were at writing them 😅

Claude 5 will write the Claude 5 safety case

Claude 4 is much better at writing AI R&D kernels compared to Claude 3.7 Sonnet

@scaling01 This is a first, but I think a good direction

Seems like a big deal