Our paper on *bridging systems* has just been published by @knightcolumbia! Joint work with @metaviv, it gives a framework for translating offline peacebuilding practices into algorithmic settings. Lots of examples + open questions.

knightcolumbia.org/content/bridgi…

🚀🚀🚀 Today we're launching Common Notes 🚀🚀🚀

A Community Notes-inspired fact checking extension for text and video. Currently in alpha.

Add to chrome: chromewebstore.google.com/detail/common-…

What's a good guess as to when the IPOs are gonna happen?

I've got you covered. Blending forecasts from @Polymarket, @Kalshi and @metaculus.

globalriskodds.com/ipo

We’re partnering with the Federation for Innovation in Democracy (FIDE North America) to host a first-of-its-kind Deliberative Facilitation Masterclass, and registration is now open!

We (@ghadfield @nickacaputo and soon @zhitzig ) are building an initiative at @JHUBloombergCtr to help society navigate the transition to powerful AI. We've all been talking about this for a while so I'll spare you the long vision statement. Right now we're looking for someone to help us build our Governing the AI Transition initiative: a talented ops person who wants to help ensure that some of the best research on successful post-AGI futures (and how to get there safely) is done in the public interest. The new GAIT director role will have a lot of autonomy, a huge upper bound in terms of impact, and the chance to work alongside Gillian and me (we'll be faculty co-directors).

I know this kind of role is truly the hardest to fill in AI safety/alignment/governance at the moment. We think being part of an AGI-pilled school of government and policy in the heart of DC is a unique opportunity. Our key values are achieving real, measurable, positive impact through rigour, independence, intellectual (and other forms of!) autonomy, a commitment to the core principles of liberal democracy (and pluralism with respect to factions within it), and a determination to build the kind of intellectual infrastructure AGI governance needs, and that we are uniquely placed to create. On top of all that, we're going to be training the next cohort of AGI governance talent, so that there are more AGI-ready researchers and generalists out there to hire!

Here's the ad. N.b. other roles (including faculty and research scientists) will be advertised soon!

hiring.jhu.edu/careers/job/11…

The AI constitutions are the most important political documents of our time. But so far they are being designed, drafted, and amended behind the closed doors of companies, with little public insight into how they work or what they mean.

Introducing Model Constitution, a publication dedicated to analyzing AI constitutions and the questions they raise and, eventually, developing an actual model AI constitution that can be used to align frontier AIs. This Model Constitution will do what the company constitutions cannot, making the public good its sole aim and integrating with the checking and balancing institutions that enable a stable and free society. It will also try to do what basically only the company constitutions have been able to: provide a usable alignment basis for advanced AI systems.

This Model Constitution will be just one perspective on how to write constitutions and design political orders for AI. Alignment is collective work to build a collective form of life. As such, the publication will also be a platform for public deliberation and input into alignment, drawing on a variety of views and on deliberative processes. I’m excited to continue collaborating with the excellent group of people in this space, including those in the labs, in external alignment research, in the Working Group on AI Constitutionalism, and elsewhere. Both the political and the technical sides of this project need great people working on them, so please reach out if it seems like Model Constitution could be useful to you.

Model Constitution is an effort to articulate what kind of political order we want in the era of AGI and to constitute that order for ourselves, for the new intelligences now coming into being, and for our collective posterity. It’s time for a Philadelphia Project on AI. I hope you’ll get involved.

Many institutions no longer serve anyone’s values.

What’s guiding them instead seems to be purposeless busywork. A kind of white noise.

In a new essay, @edelwax calls this "Value Drift" and explains why it happens, even when everyone can see what's gone wrong.

A teenager was excluded from a 2022 TikTok algorithm update designed to reduce rabbit holes of harmful content -- along with 10% of all users -- and then committed suicide after being served many depressing videos.

A tragic incident. But ethically complicated. If a platform makes a change that could plausibly reduce harm, do they have an obligation to give it to everyone immediately? If they do, it may be impossible to tell if the change worked.

My co-authors and I have previously advocated for mandatory holdout groups as a transparency measure to give public visibility into cumulative algorithm changes, good or bad. That is exactly what TikTok did here (minus the public reporting).

TikTok has since reduced holdout size (10%->1%) and duration (6 months->7 days). But this doesn't solve the underlying problem. One thing that tickles my mind is statistical protocols from medical research where you terminate one arm of the protocol when it becomes obvious that the other one is much better.

bloomberg.com/news/features/…

Powerful AI will change our institutions. Yet most of our attention is on aligning or regulating AI, while treating our institutions as fixed.

Introducing Pax Machina: a new publication about the institutions we need for powerful AI.

It is edited by @MaxKronerDale, @sebkrier, tell, @chelcott9, @synchroaphasia, @ryan_t_lowe, @edelwax, and @klingefjord.

The editorial board consists of Peter Railton, @sethlazar, @saffronhuang, @AmmannNora, @xuanalogue, @IasonGabriel, @hamandcheese, and @deanwball.

Our goal is to seed rigorous debate about what a world of humans and powerful AI could and should look like.

We will publish proposals for new institutions, counterproposals, well-justified design principles, and analyses that change how a class of institutions should be designed.

Pluralistic alignment is thriving as a research agenda yet failing at its goal: making the AI systems people actually use more pluralistic🌈

🚨New position paper: we argue adoption in deployed models should be the fields main goal & we provide a roadmap of how to get there

🧵1/

What could it mean for an AI to be "politically neutral”? And can we measure it? New paper + dataset.

We propose a defn that applies to any type of conflict: a neutral response should maximize approval on both sides of an issue, while keeping that approval balanced.

1/🧵

🚨New preprint. Many papers show AI can write fact-checks as well as humans (or better) in the lab, but very few test this in the real world.

We run the first online evaluation of AI fact-check writing with X Community Notes’ AI writer API.

Paper w. @bakkermichiel 1/

AI Note-writer Progress

Our note-writer (wholesome-raspberry-stilt) has written community notes on X with 47M views.

The cost per helpful note is about $7. You can look at our notes here:

goodheart-labs.github.io/cn-return-bot/

We are launching a new research fellowship! We are looking for 3 researchers – ideally PhD candidates or early-career postdocs* – who are passionate about AI and democratic governance.

cip.org/research-fello…

*We are open to Masters-level students!

GreenEarth is creating open source AI-driven recommender infrastructure for the ATProto ecosystem. Type a prompt, see your feed change. We are here for the users, the builders, the dreamers. Join us.
greenearthsocial.substack.com/p/introducing-…

I am an expert on community notes, mechanism design, fact-checking bot.

With increasing concern about Sora, I'm happy to talk on podcasts or read proposals. Quick thoughts:
- Centralised moderation hasn't gone great
- Solve for equilibrium
- Give people extra context

We’re watching the beginning of the end of the platform era, as agents become the new default intermediaries—Pulse and ChatGPT payments are the first big steps in this direction.

We’re going to reconfigure the platform economy. This can be done well!

My symposium on AI and Democratic Freedoms (edited with Katy Glenn Bass) is shaping up amazingly well. @random_walker and @sayashk 's already profoundly influential 'AI as Normal Technology' is in there.

So is @danielsusskind's deep and insightful investigation of what will in a world of AI agents.

And there's a great paper by @_FelixSimon_ and Sacha Altay on the actual impact of generative AI on elections. And much more that I'll be writing about later, (incl great art from @sebkrier)

Read them all at buff.ly/zTrRirO. Thanks so much to @knightcolumbia (and especially Katy Glenn Bass) for supporting this initiative.

From time to time, people suggest that human empowerment might be a useful alignment target.

This makes a lot of sense intuitively, and we have notions of empowerment (and Turner et al.'s POWER) that are plausibly suitable.

📣New preprint📣

There’s a growing trend toward building human-like AI systems with warm, friendly, and empathetic communication styles. But are these style changes just cosmetic?

Our new work shows that they can have a serious impact on model reliability & safety🧵

Very exciting to see Prosocial Media starting to come to life! @audreyt @divyasiddarth @LukeThorburn_ @edekeulenaar @JMchangama