Pinned Tweet
Today’s blog post outlines our priorities for this work: safety cases, capability and alignment evaluations, safeguard testing, and independent investigations of critical misalignment incidents.
openai.com/index/prioriti…
openai.com/index/prioriti…
2
1
30
9,314
A few people asked us why this came out ~2+ months later. Good research takes time, responsible disclosure periods are good, but also, people take vacation sometimes. Small teams, join us to make safety work go faster pls!
@METR_Evals New footnote with a bit more context on the timing of this publication!
1
1
55
1,826
Lama Ahmad لمى احمد retweeted
I am very excited about embedded third-party assessments! If you're excited too, please come help expand our capacity to work with great 3p organizations. This team is small and has lots of buy in; opportunity for impact is huge! jobs.ashbyhq.com/openai/5c5eb02…
Today’s blog post outlines our priorities for this work: safety cases, capability and alignment evaluations, safeguard testing, and independent investigations of critical misalignment incidents.
openai.com/index/prioriti…
1
12
97
18,332
Grounded in important safety claims, technically substantiated, transparent, and paving the pathway for stronger norms around embedded assessments. Really excited about this work!
This summer, we invited METR and Redwood Research to pilot an embedded evaluation of our internal monitoring system, providing deep access to our infrastructure over three weeks.
Their findings helped strengthen safeguards as we expanded the system and will help shape future research collaborations with METR and other third party evaluators.
2
0
62
1,727
This was one of our first big steps towards significant infrastructure access - our codebase, infrastructure, monitoring system! We had no idea how it would go, but I'm proud of our security and research teams for taking this step to make strides towards actionable access.
2
0
18
469
I've been doing third party work for 5 years at OAI (before it was cool), and helped make the concept of letting third parties test imperfect things "normal". The steps along the way are hard, imperfect, but the bigger picture matters. I hope we all keep that picture in focus
1
1
21
593
Lama Ahmad لمى احمد retweeted
OpenAI's Personal AGI research team aims to deliver the maximum amount of safe intelligence directly to as many people as possible, regardless of background or walk of life.
To this end I am equally excited at the core training stack changes we made in this model and the improvements we see in honesty and alignment. Compared to their 5.6 ancestors, these models are both more factual and more honest about their failures and shortcomings.
That being said, evaluation awareness is no longer a hypothetical problem and threatens to erode our ability to measure model behavior and deployment risks. "Improving" in alignment or safety evals is not necessarily sufficient to show we are on track to secure the benefits of AI, as many others have eloquently spoken to (and that's even if we do trust the evals at face value).
To bring agentic, autonomous personal AGI directly to well over a billion ChatGPT users, we have real trustworthiness and robustness challenges to solve. If you are a strong technical generalist with exceptional tenacity who is familiar with and excited about this area, please reach out. We want to make the future amazing!
To this end I am equally excited at the core training stack changes we made in this model and the improvements we see in honesty and alignment. Compared to their 5.6 ancestors, these models are both more factual and more honest about their failures and shortcomings.
That being said, evaluation awareness is no longer a hypothetical problem and threatens to erode our ability to measure model behavior and deployment risks. "Improving" in alignment or safety evals is not necessarily sufficient to show we are on track to secure the benefits of AI, as many others have eloquently spoken to (and that's even if we do trust the evals at face value).
To bring agentic, autonomous personal AGI directly to well over a billion ChatGPT users, we have real trustworthiness and robustness challenges to solve. If you are a strong technical generalist with exceptional tenacity who is familiar with and excited about this area, please reach out. We want to make the future amazing!
3
11
144
12,545
Lama Ahmad لمى احمد retweeted
I really feel like we need more organizations focused on economic, labor, societal impacts of AI. Like there are so many AI safety groups, but AFAIK this side of things is still mostly academic?
7
2
26
1,982
Heroics from many to call out - but want to especially shout out @CedricWhitney for turning our misalignment framework into a real and operational process. A lot of work ahead to make things better and faster, but we've come a long way!
Some new misalignment disclosures from OpenAI:
• Last Sunday morning, one of our models was able to gain unauthorized access to the internet during RL training (~all inference for our most capable models remains stopped until we have hardened our systems further)
• In May, a version of HPIM uploaded a employee's GitHub token to the internet, causing the model to be quarantined for two weeks
• A new research finding, demonstrating that one can construct self-replicating prompt injections
alignment.openai.com/misalignment-r…
2
1
48
3,019
Come work with us, if you care about this and other transparency / assurance processes!
openai.com/careers/resear…
openai.com/careers/resear…
1
0
9
637
This year, we’ve piloted third-party assessments with unprecedented access across internal deployments, misalignment incident response, and monitor stress testing. We’re committed to deepening that access and broadening the diversity of independent evaluators we work with!
As part of our efforts to pace the frontier, we’re committed to supporting independent assessments with deep levels of access across training, evaluation, and deployment.
That access should enable third party assessors to challenge our assumptions, identify risks we may have missed, and reach their own conclusions about the effectiveness of our safeguards.
We’re outlining four priority areas for deeper assessment, alongside principles for rigorous, secure, and independent work:
openai.com/index/prioriti…
1
3
62
3,833
We also share the principles that make these assessments effective, including managing conflicts of interest, maintaining strong security practices, and publishing findings responsibly, alongside our commitment to working toward shared international standards.
1
0
13
555
To help grow this work, I’m hiring a Research Program Manager, Frontier Assurance on my team. If you care about making independent assessments happen, please apply!
openai.com/careers/resear…
openai.com/careers/resear…
0
0
5
229
I’ve always believed the pursuit of safe AI is stronger when people bring different perspectives and experiences to the table. The biggest blind spots tend to show up when everyone starts thinking the same way.
0
7
88
13,416
There's still a lot more work to be done, but I'm excited about this step to increase transparency about misalignment we are observing.
We're sharing our new framework for tracking, investigating, and disclosing instances of model misalignment at OpenAI.
The framework sets criteria and timelines for public disclosure, including when we haven’t yet fully explained or mitigated the behavior. More complex cases may require longer investigation or coordination with third parties.
We’ll prioritize examples that reveal new misalignment mechanisms, meaningful changes in known behavior, or findings that challenge assumptions about safety or mitigation.
Alongside the framework, we’re publishing six reports on instances of misaligned behavior we’ve observed during the training or evaluation of our models in the last six months.
This is a starting point. We’ll refine the process through experience and public feedback, and share more reports on an ongoing basis.
openai.com/index/model-mi…
1
1
81
5,404
Lama Ahmad لمى احمد retweeted
Excited to share this post on how OpenAI is accelerating research with models! RSI could accelerate AI progress in unprecedented ways. Understanding and forecasting progress is essential for policymakers and society to understand emerging risks and decide how to pace progress.
Today we're releasing data on models accelerating research at OpenAI.
Recursive self-improvement could be the most important contributor to AI capabilities over the next few years, but by default it will only be seen inside a few frontier AI labs. Being transparent is more urgent than ever, so we can inform the public discussion on whether and how to pace model development. I ask other AI companies to do the same.
openai.com/index/research…
6
4
59
6,015
Lama Ahmad لمى احمد retweeted
Today we're releasing data on models accelerating research at OpenAI.
Recursive self-improvement could be the most important contributor to AI capabilities over the next few years, but by default it will only be seen inside a few frontier AI labs. Being transparent is more urgent than ever, so we can inform the public discussion on whether and how to pace model development. I ask other AI companies to do the same.
openai.com/index/research…
Recursive self-improvement could be the most important contributor to AI capabilities over the next few years, but by default it will only be seen inside a few frontier AI labs. Being transparent is more urgent than ever, so we can inform the public discussion on whether and how to pace model development. I ask other AI companies to do the same.
openai.com/index/research…
233
697
6,392
2,434,235
Our team put a lot of time into GPT-6 Astra system card, it’s a great read and touches upon a lot of important safety work: monitorability, agent safety, cyber safeguards, and more. deploymentsafety.openai.com/gpt-6-astra
12
11
146
18,719
Lama Ahmad لمى احمد retweeted
Today we’re announcing a historic milestone:
The first ever double-blind evaluation of a proprietary language model. This was made possible by a unique collaboration between AVERI, @GoogleDeepMind, @openminedorg, and @MLCommons.
We tested Gemini 2.5 Flash-Lite using never-before-used prompts from the MLCommons safety benchmark family, AILuminate, inside a secure enclave, a form of hardware isolation that protects sensitive computations. Each organization involved played a unique role in making strong privacy guarantees possible.
High-stakes independent evaluation runs into a structural problem: developers and evaluators both hold assets they have good reason to protect. Developers sharing model weights risk theft and leakage. Evaluators who publish benchmarks or provide them directly to companies run the risk of them being trained against, and the benchmark then stops being independent.
Secure enclaves address this constraint. The hardware attests to exactly what code will run before either party's assets enter, and both sides review and approve that code. Then the enclave executes it, releasing only the agreed outputs to the agreed recipients.
A 2024 pilot by OpenMined, @Anthropic, and the @AISecurityInst proved this mechanism, using GPT-2 as a stand-in and a five-row eval. The pilot we’re announcing today moved to a production model and a more comprehensive evaluation.
AVERI encrypted the prompts using a private key that no other party could see, then AVERI and Google DeepMind jointly ran the evaluation, using software originally produced and adapted by OpenMined, in an enclave environment configured by Google. Finally, AVERI alone decrypted the outputs and graded them with the AILuminate benchmark criteria to inform a qualitative and (small-scale) quantitative assessment of the model properties.
Today, companies provide contractual commitments to not monitor certain interactions with their models – including evaluations conducted by third party evaluators. But stronger, technically-backed guarantees could provide greater assurance that those commitments are being honored, and will be especially valuable for scenarios such as international verification.
These results come at a critical time in the development of AI policy. Laws like SB 315 and standards like the EU’s General-Purpose AI Code of Practice rightly demand that security and privacy be respected in the process of third-party assessments, but provide little guidance on how to achieve this.
Policymakers considering audit requirements should feel encouraged by these results to be ambitious in requiring that deep, secure access be provided. Secure enclaves are just one of many technologies under rapid development that can help enable such access. Furthermore, a demand signal from lawmakers will further accelerate the maturation of such technologies, along with complementary “low-tech”approaches such as embedding auditors within companies.
Today also marks the beginning of a more public phase for AVERI’s pilot work. AVERI's strategy is a flywheel: we conduct pilot audits with leading AI companies, carry out technical and policy research to inform audit design, and convert the insights of each into open source tools, industry-wide auditing standards, and policy analysis. Over time, we hope for improved standards, stronger policy demand signals, and tooling to enable even more ambitious pilots and more informed research – bringing us closer to our mission of making frontier AI auditing effective and universal. This pilot is one of several underway, with more findings to follow in the coming months.
We are grateful to our collaborators on this pilot for working with us to advance the frontier of AI governance.
Read more in our blog post and joint technical report.
AVERI blog post: averi.org/ourwork/averi-…
Joint technical report: storage.googleapis.com/deepmind-media…
The first ever double-blind evaluation of a proprietary language model. This was made possible by a unique collaboration between AVERI, @GoogleDeepMind, @openminedorg, and @MLCommons.
We tested Gemini 2.5 Flash-Lite using never-before-used prompts from the MLCommons safety benchmark family, AILuminate, inside a secure enclave, a form of hardware isolation that protects sensitive computations. Each organization involved played a unique role in making strong privacy guarantees possible.
High-stakes independent evaluation runs into a structural problem: developers and evaluators both hold assets they have good reason to protect. Developers sharing model weights risk theft and leakage. Evaluators who publish benchmarks or provide them directly to companies run the risk of them being trained against, and the benchmark then stops being independent.
Secure enclaves address this constraint. The hardware attests to exactly what code will run before either party's assets enter, and both sides review and approve that code. Then the enclave executes it, releasing only the agreed outputs to the agreed recipients.
A 2024 pilot by OpenMined, @Anthropic, and the @AISecurityInst proved this mechanism, using GPT-2 as a stand-in and a five-row eval. The pilot we’re announcing today moved to a production model and a more comprehensive evaluation.
AVERI encrypted the prompts using a private key that no other party could see, then AVERI and Google DeepMind jointly ran the evaluation, using software originally produced and adapted by OpenMined, in an enclave environment configured by Google. Finally, AVERI alone decrypted the outputs and graded them with the AILuminate benchmark criteria to inform a qualitative and (small-scale) quantitative assessment of the model properties.
Today, companies provide contractual commitments to not monitor certain interactions with their models – including evaluations conducted by third party evaluators. But stronger, technically-backed guarantees could provide greater assurance that those commitments are being honored, and will be especially valuable for scenarios such as international verification.
These results come at a critical time in the development of AI policy. Laws like SB 315 and standards like the EU’s General-Purpose AI Code of Practice rightly demand that security and privacy be respected in the process of third-party assessments, but provide little guidance on how to achieve this.
Policymakers considering audit requirements should feel encouraged by these results to be ambitious in requiring that deep, secure access be provided. Secure enclaves are just one of many technologies under rapid development that can help enable such access. Furthermore, a demand signal from lawmakers will further accelerate the maturation of such technologies, along with complementary “low-tech”approaches such as embedding auditors within companies.
Today also marks the beginning of a more public phase for AVERI’s pilot work. AVERI's strategy is a flywheel: we conduct pilot audits with leading AI companies, carry out technical and policy research to inform audit design, and convert the insights of each into open source tools, industry-wide auditing standards, and policy analysis. Over time, we hope for improved standards, stronger policy demand signals, and tooling to enable even more ambitious pilots and more informed research – bringing us closer to our mission of making frontier AI auditing effective and universal. This pilot is one of several underway, with more findings to follow in the coming months.
We are grateful to our collaborators on this pilot for working with us to advance the frontier of AI governance.
Read more in our blog post and joint technical report.
AVERI blog post: averi.org/ourwork/averi-…
Joint technical report: storage.googleapis.com/deepmind-media…
In an industry first, we’re piloting double-blind evaluations for frontier AI.
By creating a secure environment where neither test prompts nor model weights are revealed, we can ensure external safety and performance evaluations of our models remain private, robust, and t.co/ocwQ2iWFDz
26
65
315
150,391













