1/3 I agree with Vitalik's point: if you restrict the set of coalition-level deviations the principal has to defend against, the principal’s problem gets easier.
My question: can the restriction can be made strong enough to do the work we want from it without making the agents useless?
Useful agents cannot be informationally isolated. They need models of the world, public signals, shared data, evaluators, tools, intermediaries, and other sources of context. But once agents inhabit overlapping information environments, restricting direct communication does not eliminate coordination capacity: common signals and shared context can themselves support tacit coordination.
That creates a basic tension. The more we isolate agents to suppress tacit coordination, the less informed (and therefore less useful) they become. But once we give them enough shared context to be useful, we reopen informational pathways for coordination without explicit communication. The same information environment that enables useful action can also make tacitly coordinated deviations practically available.
So perhaps the harder problem is not simply whether agents can communicate, but where the enforcement boundary falls within the broader information environment?
That is not giving up on collusion resistance, but instead asking how to achieve it given informational interdependence.
My question: can the restriction can be made strong enough to do the work we want from it without making the agents useless?
Useful agents cannot be informationally isolated. They need models of the world, public signals, shared data, evaluators, tools, intermediaries, and other sources of context. But once agents inhabit overlapping information environments, restricting direct communication does not eliminate coordination capacity: common signals and shared context can themselves support tacit coordination.
That creates a basic tension. The more we isolate agents to suppress tacit coordination, the less informed (and therefore less useful) they become. But once we give them enough shared context to be useful, we reopen informational pathways for coordination without explicit communication. The same information environment that enables useful action can also make tacitly coordinated deviations practically available.
So perhaps the harder problem is not simply whether agents can communicate, but where the enforcement boundary falls within the broader information environment?
That is not giving up on collusion resistance, but instead asking how to achieve it given informational interdependence.
It's possible that a killer app of adversarial governance mechanism design theory will end up being AI safety.
Compare:
vitalik.eth.limo/general/2020/0…
aiprospects.substack.com/p/preventing-a…
There's a deep duality between the two environments. Both are about a less-sophisticated principal trying to get ideal outcomes from a set of more-sophisticated agents: in the first case, the principal is a static algorithm and the agents are humans, in the second case, the principal is humans plus weaker LLMs and the agents are stronger LLMs.
A key finding was that you can achieve much better outcomes if you can guarantee limits to how much agents can collude (see: Nash equilibria being abundant, vs. cooperative-game-theory "cores" often being empty). That argument should naturally transfer to this new setting.
1
3
3
1,235
2/3 This question is not new but traces back to our earlier work.
In DeSoc, @glenweyl @VitalikButerin and I argued that mechanism design cannot simply begin with atomized, independent participants and then decide how much coordination to permit.
People arrive with pre-existing cooperation, overlapping affiliations, and social dependencies that can produce “innocent over-coordination” before intentional collusion even enters the picture. Correlated behavior need not begin with direct communication at the moment of action; it can reflect relationships and affiliations in which participants are already embedded.
That is why we proposed “correlation discounts”: overlapping affiliations could serve as an ex ante signal that nominally distinct participants were not socially independent, so agreement among them should not be weighted as though it represented the same breadth of social difference as agreement across more distant groups. [1]
The Idena case study with @_paulaberman @cryptoidentity in Compressed to 0 illustrated the point empirically. The protocol could verify distinct flesh-and-blood humans and treat them as separate players in its formal mechanism, but it could not infer their relationships from the on-chain game alone. The same observable transaction pattern could be consistent with either voluntary cooperation or off-chain puppeteering. Distinguishing between them required looking at the underlying distribution of information and control in meatspace: who understood the protocol, who held the keys, who directed whom, and on what terms. [2]
So Compressed to 0 made concrete what DeSoc had raised theoretically: formal distinctness did not tell you how socially or informationally independent participants actually were. And tightening the formal enforcement boundary did not necessarily eliminate the underlying relationships. (Measures that made on-chain vote-buying harder could push the interaction off-chain into “meatspace.” ) The mechanism could change where coordination occurred without eliminating the relationships that made it possible.
That seems directly relevant to AI. You can prohibit direct model-to-model communication, but useful agents still have to inhabit an information environment. If they share a base model, training corpus, evaluator, retrieval environment, tool ecosystem, public signals, or other common context, part of the informational substrate shaping their strategies remains shared.
Common information is not collusion per se, but it can support tacit coordination or “innocent over-coordination.” So I’m not convinced that restricting direct communication is enough to constrain the coordination capacity behind the deviations we care about.
The harder question seems to be about informational interdependence: which forms are necessary for usefulness, which create dangerous coordination capacity, and how to make that distinction legible and governable?
[1] papers.ssrn.com/sol3/papers.cf…
[2] papers.ssrn.com/sol3/papers.cf…
In DeSoc, @glenweyl @VitalikButerin and I argued that mechanism design cannot simply begin with atomized, independent participants and then decide how much coordination to permit.
People arrive with pre-existing cooperation, overlapping affiliations, and social dependencies that can produce “innocent over-coordination” before intentional collusion even enters the picture. Correlated behavior need not begin with direct communication at the moment of action; it can reflect relationships and affiliations in which participants are already embedded.
That is why we proposed “correlation discounts”: overlapping affiliations could serve as an ex ante signal that nominally distinct participants were not socially independent, so agreement among them should not be weighted as though it represented the same breadth of social difference as agreement across more distant groups. [1]
The Idena case study with @_paulaberman @cryptoidentity in Compressed to 0 illustrated the point empirically. The protocol could verify distinct flesh-and-blood humans and treat them as separate players in its formal mechanism, but it could not infer their relationships from the on-chain game alone. The same observable transaction pattern could be consistent with either voluntary cooperation or off-chain puppeteering. Distinguishing between them required looking at the underlying distribution of information and control in meatspace: who understood the protocol, who held the keys, who directed whom, and on what terms. [2]
So Compressed to 0 made concrete what DeSoc had raised theoretically: formal distinctness did not tell you how socially or informationally independent participants actually were. And tightening the formal enforcement boundary did not necessarily eliminate the underlying relationships. (Measures that made on-chain vote-buying harder could push the interaction off-chain into “meatspace.” ) The mechanism could change where coordination occurred without eliminating the relationships that made it possible.
That seems directly relevant to AI. You can prohibit direct model-to-model communication, but useful agents still have to inhabit an information environment. If they share a base model, training corpus, evaluator, retrieval environment, tool ecosystem, public signals, or other common context, part of the informational substrate shaping their strategies remains shared.
Common information is not collusion per se, but it can support tacit coordination or “innocent over-coordination.” So I’m not convinced that restricting direct communication is enough to constrain the coordination capacity behind the deviations we care about.
The harder question seems to be about informational interdependence: which forms are necessary for usefulness, which create dangerous coordination capacity, and how to make that distinction legible and governable?
[1] papers.ssrn.com/sol3/papers.cf…
[2] papers.ssrn.com/sol3/papers.cf…
1
0
1
209
3/3 This is where my later work on communication currencies starts.
If useful agents necessarily remain embedded in shared information environments, perhaps the relevant design space is one of graded constraints rather than hard communication boundaries alone.
Communication currencies try to introduce those gradients through context. In DeSoc, credentials and overlapping affiliations served as proxies for pre-existing cooperation and social dependence. But those credentials were themselves emergent social signposts of prior relationships and communication.
Communication currencies move a layer upstream: irrevocable stake and transferable currency help locate agents within the communication network itself, representing degrees of participation in and commitment to communication channels. That gives a richer and more dynamic (ex ante and ex post) picture of likely informational overlap.
The question then shifts from simply whether two agents can talk to **how much** information can flow between them, at what depth, under what conditions, and at what cost given the risk.
More socially proximate agents with shared stake (skin in the game) can access richer, higher-bandwidth contextual information; more distant agents interact through narrower, more public channels. On the control side, similarly situated or informationally redundant agents can be correlation-discounted in influence, while more informationally distinct agents can receive bridging bonuses.
That makes subsidiarity more than a decentralization principle; it becomes an enforcement requirement (hence also “community currencies.”) Scrutiny is pushed toward the places where information is richest and monitoring costs are lowest, rather than concentrated in a global monitor that has to surveil everyone (and, in doing so, risks flattening the very context-sensitive distinctions it is trying to discern.)
So the design objective I’m circling around is collusion resistance under informational interdependence.
The goal is still to restrict the coalition-level deviation set the principal has to defend against. My extension is to ask whether some of those constraints can operate in a graded way: information access and communication (and with them coordination capacity) can vary in bandwidth, cost, and consequence depending on social distance and risk.
So rather than relying only on fixed boundaries around communication, perhaps enforcement can be embedded more deeply in the communication architecture itself?
I don’t know whether communication currencies get this right….But if informational interdependence is unavoidable, I think this is a design space worth exploring.
[3] papers.ssrn.com/sol3/papers.cf…
If useful agents necessarily remain embedded in shared information environments, perhaps the relevant design space is one of graded constraints rather than hard communication boundaries alone.
Communication currencies try to introduce those gradients through context. In DeSoc, credentials and overlapping affiliations served as proxies for pre-existing cooperation and social dependence. But those credentials were themselves emergent social signposts of prior relationships and communication.
Communication currencies move a layer upstream: irrevocable stake and transferable currency help locate agents within the communication network itself, representing degrees of participation in and commitment to communication channels. That gives a richer and more dynamic (ex ante and ex post) picture of likely informational overlap.
The question then shifts from simply whether two agents can talk to **how much** information can flow between them, at what depth, under what conditions, and at what cost given the risk.
More socially proximate agents with shared stake (skin in the game) can access richer, higher-bandwidth contextual information; more distant agents interact through narrower, more public channels. On the control side, similarly situated or informationally redundant agents can be correlation-discounted in influence, while more informationally distinct agents can receive bridging bonuses.
That makes subsidiarity more than a decentralization principle; it becomes an enforcement requirement (hence also “community currencies.”) Scrutiny is pushed toward the places where information is richest and monitoring costs are lowest, rather than concentrated in a global monitor that has to surveil everyone (and, in doing so, risks flattening the very context-sensitive distinctions it is trying to discern.)
So the design objective I’m circling around is collusion resistance under informational interdependence.
The goal is still to restrict the coalition-level deviation set the principal has to defend against. My extension is to ask whether some of those constraints can operate in a graded way: information access and communication (and with them coordination capacity) can vary in bandwidth, cost, and consequence depending on social distance and risk.
So rather than relying only on fixed boundaries around communication, perhaps enforcement can be embedded more deeply in the communication architecture itself?
I don’t know whether communication currencies get this right….But if informational interdependence is unavoidable, I think this is a design space worth exploring.
[3] papers.ssrn.com/sol3/papers.cf…
0
0
2
92
Most people still don’t see that anarcho-capitalism risks culminating in an AGI dictatorship whose political economy is, ironically, closest to communism:
production and distribution cease to be coordinated through markets and instead become separate problems centrally planned.
One way out is to not abandon property rights, but properly scope them (i.e. stop treating datasets as private goods or public goods, but as plural goods held in partial common ownership by participants to different degrees of their contribution.)
production and distribution cease to be coordinated through markets and instead become separate problems centrally planned.
One way out is to not abandon property rights, but properly scope them (i.e. stop treating datasets as private goods or public goods, but as plural goods held in partial common ownership by participants to different degrees of their contribution.)
3
0
10
778
Puja Ohlhaver retweeted
Folks, this proposal is the big one!!!
Someday, it will be remembered as the first rumbling of a tectonic shift in how the law protects our rights to the information we co-produce every day.
Remember this phrase: Right to Digital Association.
This @FamStudies proposal sketches what our legal systems can and must do to put a tourniquet on technology’s dissolution of family, community, and other forms of human association.
Big Tech is strong-arming us into a society we will not recognize. So our politics needs to get creative: building amazing new alliances around protecting a future worth believing in.
It won’t be easy. But it will work.
congrats @RealDCochrane @salome_viljoen_ @FamStudies @RadxChange @lace31692
Someday, it will be remembered as the first rumbling of a tectonic shift in how the law protects our rights to the information we co-produce every day.
Remember this phrase: Right to Digital Association.
This @FamStudies proposal sketches what our legal systems can and must do to put a tourniquet on technology’s dissolution of family, community, and other forms of human association.
Big Tech is strong-arming us into a society we will not recognize. So our politics needs to get creative: building amazing new alliances around protecting a future worth believing in.
It won’t be easy. But it will work.
congrats @RealDCochrane @salome_viljoen_ @FamStudies @RadxChange @lace31692
Big Tech’s power is underwritten by their near total capture of our data. Tech companies capture information on everything: what we like on Instagram, the places we visit, what we do—even how we work and the culture we help create.
All that data is fueling hyper-addictive digital products and AI systems that threaten to erode or displace human relationships, social trust, meaningful work, and agency.
The good news: there’s a fix 👇 h/t @FamStudies
0
7
16
1,416
1/ The YIMBY–NIMBY housing debate is one of the sharpest political pain points in cities today, and one where conventional consultation can take months/years while reproducing the same divisions.
What makes this work by @RadxChange especially striking is the combination of speed and methodological innovation.
What makes this work by @RadxChange especially striking is the combination of speed and methodological innovation.
We partnered with AMPLIFY Australia to put ⿻ plural tech to the test on one of the most entrenched debates in Australian public life: Sydney's density fight. Beyond NIMBY vs YIMBY, what we found is that even the most engaged partisans hold conditions the other side can meet.
Check the key findings 👉 radicalxchange.org/density-done-w…
🧵
2
5
9
2,239
6/ At representative scale, Uncommon Ground surfaced a different cluster, largely around affordability:
-defining affordable housing as no more than 30% of household income
-incentives for developers who sell to owner-occupiers rather than investors
-prioritizing unhoused people
-even the more radical proposition that everyone should have the opportunity to buy a first home before others buy a second.
-defining affordable housing as no more than 30% of household income
-incentives for developers who sell to owner-occupiers rather than investors
-prioritizing unhoused people
-even the more radical proposition that everyone should have the opportunity to buy a first home before others buy a second.
1
0
2
67
7/ These weren’t consensus positions.
That’s the point!
They were cases where the established camps stopped predicting the answer and potential starting points for political coalitions we don’t see when we only map the familiar divide.
Fantastic work @_paulaberman @RadxChange!
That’s the point!
They were cases where the established camps stopped predicting the answer and potential starting points for political coalitions we don’t see when we only map the familiar divide.
Fantastic work @_paulaberman @RadxChange!
1
0
2
77
6/ At representative scale, Uncommon Ground surfaced a different cluster, largely around affordability:
-defining affordable housing as no more than 30% of household income
-incentives for developers who sell to owner-occupiers rather than investors
-prioritizing unhoused people
-even the more radical proposition that everyone should have the opportunity to buy a first home before others buy a second.
-defining affordable housing as no more than 30% of household income
-incentives for developers who sell to owner-occupiers rather than investors
-prioritizing unhoused people
-even the more radical proposition that everyone should have the opportunity to buy a first home before others buy a second.
1
0
1
41
7/ These weren’t consensus positions.
That’s the point!
They were cases where the established camps stopped predicting the answer and potential starting points for political coalitions we don’t see when we only map the familiar divide.
Fantastic work @_paulaberman @RadxChange!
That’s the point!
They were cases where the established camps stopped predicting the answer and potential starting points for political coalitions we don’t see when we only map the familiar divide.
Fantastic work @_paulaberman @RadxChange!
0
0
1
42
Puja Ohlhaver retweeted
The idea first started taking shape when @pujaohlhaver proposed to create a "decentralized deliberation standard" during a call with @yutingj_.
After that I took two weeks to launch dds.xyz.
Almost immediately, it attracted a lot of attention from many communities
After that I took two weeks to launch dds.xyz.
Almost immediately, it attracted a lot of attention from many communities
1
3
5
124
Puja Ohlhaver retweeted
I love how the nixo-gate "deletion scandal" is 100% on track to get people thinking about correlation discounts that @pujaohlhaver was talking about years ago 😂
Yes, alignment matters, just not in the way you've understood the meme so far!
Yes, alignment matters, just not in the way you've understood the meme so far!
1/ Thrilled to release a first empirical study about Proof of Personhood: “Compressed to 0: The Silent Strings of Personhood" with my esteemed co-authors Mikhail, @cryptoidentity, founder of Idena Protocol, and @_paulaberman.
papers.ssrn.com/sol3/papers.cf…
Why does PoP matter?
5
1
15
2,033