Pinned Tweet
I'm still looking for a postdoc to join my group at @ELLISInst_Tue and @MPI_IS! We offer very attractive working conditions (no teaching load, no bureaucracy, very competitive salary). Priority areas: scalable oversight, loss of control, RSI.
If you're interested, fill out this form forms.gle/uu1UrN8RQrSy8w…!
If you're interested, fill out this form forms.gle/uu1UrN8RQrSy8w…!
5
33
172
24,733
many people think that PostTrainBench is purely a capability eval... however, Muse Spark 1.3 system card (correctly) puts it in the Loss of Control section of key catastrophic-risk evals! this aligns well with our original motivation for creating PTB.
research.meta.ai/static/muse-sp…
research.meta.ai/static/muse-sp…
0
0
25
1,007
Fun fact: 4 benchmarks from Tübingen were used by Meta to report the performance of Muse Spark 1.3 in their system card:
- PostTrainBench
- Skill-Inject
- RepoInject v1 (from @expsecai)
- BrowserInject v1 (from @expsecai)
research.meta.ai/static/muse-sp…
- PostTrainBench
- Skill-Inject
- RepoInject v1 (from @expsecai)
- BrowserInject v1 (from @expsecai)
research.meta.ai/static/muse-sp…
3
6
69
3,329
it was great to collab on this via @expsecai! genuinely impressed by how quickly Meta TBD is moving.
8/N (Third-party Evaluations). Greatly appreciate the pre-launch safety testing completed by 5 third-party red teams (Scale AI @scale_AI , Handshake AI @joinHandshake , Gray Swan AI @GraySwanAI , Exponential Security Labs @expsecai and Irregular @Irregular) **particularly in the cyber and adversarial robustness** sections of this report. This results in 6 benchmarks and 1 report.
0
1
19
1,024
Maksym Andriushchenko retweeted
It was great to work with @AIatMeta folks, and they are building some seriously good models. Happy to have contributed a bit in this.
We are proud to share that our two new benchmarks, RepoInject v1 and BrowserInject v1, were used by Meta TBD Labs to report the robustness of Muse Spark 1.3 to prompt injections.
The robustness of Muse Spark 1.3 is comparable to that of GPT-5.6 Sol—congrats to the team!
1/4
0
1
5
226
Maksym Andriushchenko retweeted
Excited to share these news! Was awesome to work with the folks @AIatMeta, the speed at which they are building is crazy. Looking forward to what is next!
We are proud to share that our two new benchmarks, RepoInject v1 and BrowserInject v1, were used by Meta TBD Labs to report the robustness of Muse Spark 1.3 to prompt injections.
The robustness of Muse Spark 1.3 is comparable to that of GPT-5.6 Sol—congrats to the team!
1/4
0
3
12
813
We’ve been cooking at Exponential!
We’ve heard that prompt injections are solved. Well, not yet…
Reach out if you are interested in evaluations on hard, unsaturated security and safety benchmarks!
We’ve heard that prompt injections are solved. Well, not yet…
Reach out if you are interested in evaluations on hard, unsaturated security and safety benchmarks!
We are proud to share that our two new benchmarks, RepoInject v1 and BrowserInject v1, were used by Meta TBD Labs to report the robustness of Muse Spark 1.3 to prompt injections.
The robustness of Muse Spark 1.3 is comparable to that of GPT-5.6 Sol—congrats to the team!
1/4
1
4
29
1,520
Maksym Andriushchenko retweeted
🎉 Skill-Inject made it into the Muse Spark system card!🎉
Muse Spark 1.3 is the first model we released in Muse, and we evaluate its capabilities and behaviors in those agentic environments.
research.meta.ai/static/muse-sp…
1
2
30
1,379
Muse Spark 1.3 achieves 0.6% attack success rate on our Skill-Inject! good to see substantial progress on highly contextual prompt injections.
Muse Spark 1.3 is the first model we released in Muse, and we evaluate its capabilities and behaviors in those agentic environments.
research.meta.ai/static/muse-sp…
2
2
29
2,044
Maksym Andriushchenko retweeted
in a way, every good benchmark is now a meta benchmark target for auto-research agents (as exemplified by @maksym_andr s posttrainbench)
0
1
7
514
Maksym Andriushchenko retweeted
Insane alpha in going to Tübi for anyone academia pilled in AI rn (incl PhD students)
To put it in perspective more directly, I think anyone going to any other school (except maybe like 3 in the US) is probably better off going to Tübingen, all else equal
To put it in perspective more directly, I think anyone going to any other school (except maybe like 3 in the US) is probably better off going to Tübingen, all else equal
If you're considering academia, Tübingen is an amazing place to establish your research group and do world-class research!
The deadline to aply is Dec 13. Feel free to reach out to me if you have any questions regarding this position.
4
8
144
16,374
Maksym Andriushchenko retweeted
New paper!
How do you get a model to tell you something that it doesn't want to?
To study this we trained models that sandbag: they intentionally underperform, here by hiding code vulnerabilities they have found, from their overseer
🧵
How do you get a model to tell you something that it doesn't want to?
To study this we trained models that sandbag: they intentionally underperform, here by hiding code vulnerabilities they have found, from their overseer
🧵
2
21
98
3,698
if you ask ChatGPT, Claude, or Grok, their stated p(doom) will be around 10% regardless of model size and reasoning effort. it's interesting how independently trained models converge to the same ~consensus answer for an open-ended question.
i think the same applies very broadly: avoid opinionated takes, which is what makes humans (still) unique.
i think the same applies very broadly: avoid opinionated takes, which is what makes humans (still) unique.
2
0
22
1,411
If you're considering academia, Tübingen is an amazing place to establish your research group and do world-class research!
The deadline to aply is Dec 13. Feel free to reach out to me if you have any questions regarding this position.
The deadline to aply is Dec 13. Feel free to reach out to me if you have any questions regarding this position.
🚀 The ELLIS Institute Tübingen is hiring Principal Investigators as Hector Endowed Fellows!
Enjoy globally competitive packages, full academic freedom & possible co-appointments w/ @MPI_IS and the Tübingen AI Center (@uni_tue).
📍 Tübingen, 🇩🇪
📅 Apply by Dec 13
🔗 http#MachineLearning #Hiring
1
3
88
13,297
it's remarkable how naive extrapolation can work exceptionally well in some cases (scaling laws, time horizons) and completely break down in others (cyber vulnerabilities, high-severity agentic incidents)! most disagreements about future AI progress seem to stem from this.
4
0
27
3,176
Maksym Andriushchenko retweeted
arxiv.org/abs/2603.24511
arxiv.org/abs/2609.30266
stolen-thoughts.com
arxiv.org/abs/2607.19321
Tarxiv.org/abs/2607.19321ym's group.
@sahar_abdelnabi 's group@sahar_abdelnabi interesting work (arxiv.org/abs/2605.28591, arxiv.org/abs/2605.28591).arxiv.org/abs/2609.39863lear that it is not just Berkeley + Oxford + Tubingen, there is also work happening outside Tubingen in Europe (e.g., my own work which I did mostly at Cambridge in UK uzman-anwar.github.io, Florian Tramer's group at ETH Zurich) and US, e.g., @davidbau 's lab at NorthEastern (baulab.info).
arxiv.org/abs/2609.30266
stolen-thoughts.com
arxiv.org/abs/2607.19321
Tarxiv.org/abs/2607.19321ym's group.
@sahar_abdelnabi 's group@sahar_abdelnabi interesting work (arxiv.org/abs/2605.28591, arxiv.org/abs/2605.28591).arxiv.org/abs/2609.39863lear that it is not just Berkeley + Oxford + Tubingen, there is also work happening outside Tubingen in Europe (e.g., my own work which I did mostly at Cambridge in UK uzman-anwar.github.io, Florian Tramer's group at ETH Zurich) and US, e.g., @davidbau 's lab at NorthEastern (baulab.info).
3
3
38
2,342
Maksym Andriushchenko retweeted
This next paper is a bit quirky, but here goes:
Introspection is hard to study in both humans and LLMs. But, we do have internal senses that are verifiable, like proprioception (the sense of where your arm is right now).
What would proprioception look like in language models?
to the sense of position, a model could verbalize which expert activated in a certain layer for any previous token.
It turns out that LLMs can learn a range of such proprioception skills where they understand and report on their internals. We can calibrate and score these reports, and evaluate whether training for proprioception changes model behaviors.
Introspection is hard to study in both humans and LLMs. But, we do have internal senses that are verifiable, like proprioception (the sense of where your arm is right now).
What would proprioception look like in language models?
to the sense of position, a model could verbalize which expert activated in a certain layer for any previous token.
It turns out that LLMs can learn a range of such proprioception skills where they understand and report on their internals. We can calibrate and score these reports, and evaluate whether training for proprioception changes model behaviors.
14
69
468
22,501
Maksym Andriushchenko retweeted
Some thoughts on reviewing: spylab.ai/blog/reviewing/
First for @satml_conf where we trialed an early reject phase which cut submissions by ~40%.
Then more generally about how not everything deserves peer-review, and why we should reward writing things that people actually read.
First for @satml_conf where we trialed an early reject phase which cut submissions by ~40%.
Then more generally about how not everything deserves peer-review, and why we should reward writing things that people actually read.
8
38
282
41,311
MiniMax M3.1 Flash reaches 39.3% on PostTrainBench-v1.1 which is very close to the 40.4% of Opus 5.
seems like a strong release! congrats to the MiniMax team.
seems like a strong release! congrats to the MiniMax team.
MiniMax M3.1 Flash: Space Performance + Bunny Speed
🚀🐰 Yes, Space Bunny - which has topped global leaderboards with 200T+ tokens - was an earlier version of M3.1 Flash.
💰Redefining the performance economics of agentic AI. M3.1 Flash achieves frontier-level intelligence s Cowork (Automation-Bench: 62%; ALE: 30%), Coding (DeepSWE: 74%) and Visual Reasoning (BabyVision: 91%), at 1/3 the cost of M3 and 3x the speed.
API pricing (flat rate): $0.15 / $0.60 per 1M input/output tokens, with cache reads at $0.015 / 1M tokens.
Model weights will arrive on Oct 23.
API: platform.minimax.io
M Plan: minimax.io/m-plan
MiniMax Code for desktop: agent.minimax.io/download
0
0
10
1,033
Maksym Andriushchenko retweeted
We are looking for talented people to join us at @expsecai!
If you are excited about helping to shape the future of AI safety & security, make sure to apply or shoot me a message!
0
2
10
1,642
Maksym Andriushchenko retweeted
@evanjayconway @kotekjedi_ml @maksym_andr @full__rank @XinCynthiaChen @jonasgeiping sorry for the late reply. we've published the checkpoints here: huggingface.co/aisa-group/tra…
1
1
11
741