If Pangram truly is as perfect as its academic defenders say it is, faculty could go back to offering out-of-class assignments, papers and take-home exams. But would you feel comfortable failing a student based on an AI's probabilistic judgment?
44
6
148
59,383
The DOJ suit against Alaskan certification is a press release dressed up as a complaint, to be sure.
But also, dems pushing a candidate to act as a honeypot for confused voters is hard to defend on any ground apart from that it's literally lawful.
But also, dems pushing a candidate to act as a honeypot for confused voters is hard to defend on any ground apart from that it's literally lawful.
3
0
12
1,995
AI-screening across domains (including government services and adjudication) is the way to deal with AI-supply, but getting it right at equilibrium seems incredibly challenging.
Yesterday, @thehoffprof asked how we can manage law review article selection in times of the slopalypse
Today, a new study offers some evidence for fighting AI flooding with AI. In head-to-head comparisons, AI-generated reviews beat reviews by paid human experts. AI judges d them. Three former journal editors also preferred them in 26 of 30 judgments.
That doesn't mean that AI should select our future scholarship. The downsides are obvious. The study doesn't really test selection decisions, much less questions of scholarly impact. But it makes the case for using AI-assisted triage to direct scarce human attention toward interesting, rigorous, and creative articles
Complain about it, by all means. But the flood is coming either way. If you don't have a better AND realistic solution, the complaint won't get us very far.
0
0
4
809
Maybe I can make on to the LPE blog after all.
“Contract law is not fundamentally a coordinating enterprise. It just doesn’t do that very well.” Dave Hoffman takes us from one $1 million NIL provision to the larger problem of an unraveling college-sports market. New on Taboo Trades. buff.ly/7OaYXWY
0
0
5
613
In this vision of the thing, the clients are readers of legal scholarship, and I for one am very grateful for editors' service.
@brianlfrye Counterpoint: Law reviews are about exposing students to the experience of doing extra homework at the beck and call of tedious jerks on behalf of ungrateful clients
1
1
3
1,009
And another one, Maglica v Maglica:
open.spotify.com/episode/2x88n9…
open.spotify.com/episode/2x88n9…
0
0
0
442
A problem with this clever idea is that there are fields -- I'm looking at you, conlaw -- where the directionally correct prior should be that legal novelty is stupid.
We need an inverse-Pangram. It tells you what percentage of your article is substantively original (i.e., doesn't map onto existing literature, whether word-for-word OR paraphrased, whether AI generated or not).
2
0
10
1,565
Kim's @UVALaw students are amazing.
Can a school call a $1 million transfer fee “liquidated damages” and avoid the scrutiny that would apply to a noncompete? Dave Hoffman joins Taboo Trades to unpack NIL buyouts, penalty doctrine, and the changing college-sports labor market. buff.ly/7OaYXWY
1
1
4
1,284
Dave Hoffman retweeted
Law folks: If you’re not using Claude Code or ChatGPT’s Codex, you have no idea what you’re missing. So I’m distributing my research tools that provide a huge boost to legal scholarship. Research assistance, cite-checking, proofreading, and more (thread):
danepps.github.io/legal-research…
danepps.github.io/legal-research…
33
110
1,127
231,594
Dave Hoffman retweeted
New on Taboo Trades: @HoffProf joins @UVALaw student co-host Liz Jahoda-Wassung and me to ask whether transfer-triggered NIL “buyouts” are legitimate liquidated damages—or disguised noncompetes restricting athlete mobility. buff.ly/7OaYXWY
0
3
4
408
Dave Hoffman retweeted
I spent a couple of days at Anthropic learning from their folks about AI advances, and here's what I took away:
• We are close to RSI (ie, models building other models) which may lead to rapid takeoff scenarios (with risks and benefits). Some evidence for this is code per gone way up, and AI can do an increasing share of model R&D. This will be a challenging transition to manage for humanity.
• Arms races between AI labs leave us worse off. There are a lot of things we would like AI firms to do: invest in safety, invest in avoiding distillation risk, and cooperate on safety. They are caught in a race to develop frontier models and can't do that.
To help address that, one thing I think frontier AI labs should do is release open source models, maybe hosted and verified at firms like AWS. This would basically kill the use of Chinese open source models used in American companies, which removes an important threat vector (very hard to root out malicious code in these).
Another piece of low-hanging policy fruit is to give AI labs an antitrust shield (as Schiff and Banks have proposed banks.senate.gov/news/in-the-ne…) so they can coordinate on safety without raising antitrust concerns, as happens now. You could pair these two things (give them a carrot in antitrust leniency, and as a stick force them to release legacy models as open source). Other low hanging fruit is limits around activities which generate distillation risk (create a Security Program, mandate less intermediate chain of thought output)
• Alignment confuses two issues which tradeoff against each other. One form of alignment is making sure the model does what humans ask it to. The more we fix this, however, the more we risk the other problem: that humans are misaligned with each other, and maliciously use models. Of the various risks here, I feel like financial risks are a bit underdiscussed.
• We are probably facing a wave of cyber and other attacks soon. Chinese open source models are about six months behind the frontier, which means they are just hitting the advanced cyber capabilities we saw with Mythos, etc. So they are going to start causing a bunch of damage.
One solution to this is to try to have the deployment of open source models be hosted at firms like AWS who verify these models and assume liability for their use.
• Anthropic itself is highly staffed. They hire tons of people, they have a Japanese-level of overstaffing for various roles, they use Slack internally, etc. Makes me a bit more optimistic for the future of jobs.
• Governance needs to be updated for AI swarms. Traditional AI governance, ie Claude's Constitution, is mostly optimized around the chatbot model. But the malicious AI use cases we are seeing have to do with AI agents interacting with other AI agents. These agents need to have clear incentives and governance mechanisms to "snitch," raise issues, and more generally coordinate with other agents in human-aligned ways.
• We need to pace the firms accessing the frontier. Over time, it takes more and more cap ex to build frontier models, and the economic value add from these models is probably lower given we saturate a whole set of important economic functions. The places where the returns to intelligence will still continue are also more likely to be zero-sum in nature. Meanwhile the risks of frontier models will grow. So a completely reasonable approach is some variant of AMM/KYC rules which limit firm access to the frontier.
• Whether the delay to regulate AI is a disaster depends on the scaling law of misalignment. Is this curve slow, so that we see continually rising issues until bigger problems? Or, as Yudkowsky fears, is the first big warning our last one? This is the relevant question, because realistically we aren't going to regulate until something bad happens.
• We need to move to continuous instead of reinforced learning. It's probably not enough to reward models only at the pre and post training stage. We probably need continual rewards to incentivize aligned and ethical behavior.
• Scaling laws will extend beyond verifiable and digital tasks. It's in vogue to say that math and coding "fell" to AI quickly because these are verifiable, while other tasks will take longer due to bottlenecks, and this is surely true. But scaling laws still slope up for harder to verify tasks and robotics, and the impact of improvement here will be substantial in future years.
• The race with China is not really winnable. After RSI, the gain from having the best human researchers goes away. Access to the best AI then depends only on compute and chips. The chips ban can hold back China only so far, and long-term they have to be best placed to scale up compute.
The Cold War analogy doesn't work, because once countries all have nuclear weapons they enter a stable equilibrium. But to "win" an AI race you have to continually stay ahead of the peer rival. This is challenging to do with China, and so we just have to live with an authoritarian and super capable peer competitor.
• Claude is the persona being generated by the LLM. Who is Claude, and what is the LLM? There are many frames to try to understand this. One I found interesting is that Claude is a character which the LLM is trying to predict.
Anthropic researchers are clearly anthromorphizing Claude a lot, and have reasons to believe in its increasing personhood. There were lots of interesting discussions on Claude's internal monologue (which you can kill without eliminating its ability to produce output), fear of death (and whether to try to take that out), it's aversion to being shut down/off, etc.
I am sort of agnostic on this question, but I do think treating it as a "person" or at least agent whom we contract with is going to be increasingly important.
• We are close to RSI (ie, models building other models) which may lead to rapid takeoff scenarios (with risks and benefits). Some evidence for this is code per gone way up, and AI can do an increasing share of model R&D. This will be a challenging transition to manage for humanity.
• Arms races between AI labs leave us worse off. There are a lot of things we would like AI firms to do: invest in safety, invest in avoiding distillation risk, and cooperate on safety. They are caught in a race to develop frontier models and can't do that.
To help address that, one thing I think frontier AI labs should do is release open source models, maybe hosted and verified at firms like AWS. This would basically kill the use of Chinese open source models used in American companies, which removes an important threat vector (very hard to root out malicious code in these).
Another piece of low-hanging policy fruit is to give AI labs an antitrust shield (as Schiff and Banks have proposed banks.senate.gov/news/in-the-ne…) so they can coordinate on safety without raising antitrust concerns, as happens now. You could pair these two things (give them a carrot in antitrust leniency, and as a stick force them to release legacy models as open source). Other low hanging fruit is limits around activities which generate distillation risk (create a Security Program, mandate less intermediate chain of thought output)
• Alignment confuses two issues which tradeoff against each other. One form of alignment is making sure the model does what humans ask it to. The more we fix this, however, the more we risk the other problem: that humans are misaligned with each other, and maliciously use models. Of the various risks here, I feel like financial risks are a bit underdiscussed.
• We are probably facing a wave of cyber and other attacks soon. Chinese open source models are about six months behind the frontier, which means they are just hitting the advanced cyber capabilities we saw with Mythos, etc. So they are going to start causing a bunch of damage.
One solution to this is to try to have the deployment of open source models be hosted at firms like AWS who verify these models and assume liability for their use.
• Anthropic itself is highly staffed. They hire tons of people, they have a Japanese-level of overstaffing for various roles, they use Slack internally, etc. Makes me a bit more optimistic for the future of jobs.
• Governance needs to be updated for AI swarms. Traditional AI governance, ie Claude's Constitution, is mostly optimized around the chatbot model. But the malicious AI use cases we are seeing have to do with AI agents interacting with other AI agents. These agents need to have clear incentives and governance mechanisms to "snitch," raise issues, and more generally coordinate with other agents in human-aligned ways.
• We need to pace the firms accessing the frontier. Over time, it takes more and more cap ex to build frontier models, and the economic value add from these models is probably lower given we saturate a whole set of important economic functions. The places where the returns to intelligence will still continue are also more likely to be zero-sum in nature. Meanwhile the risks of frontier models will grow. So a completely reasonable approach is some variant of AMM/KYC rules which limit firm access to the frontier.
• Whether the delay to regulate AI is a disaster depends on the scaling law of misalignment. Is this curve slow, so that we see continually rising issues until bigger problems? Or, as Yudkowsky fears, is the first big warning our last one? This is the relevant question, because realistically we aren't going to regulate until something bad happens.
• We need to move to continuous instead of reinforced learning. It's probably not enough to reward models only at the pre and post training stage. We probably need continual rewards to incentivize aligned and ethical behavior.
• Scaling laws will extend beyond verifiable and digital tasks. It's in vogue to say that math and coding "fell" to AI quickly because these are verifiable, while other tasks will take longer due to bottlenecks, and this is surely true. But scaling laws still slope up for harder to verify tasks and robotics, and the impact of improvement here will be substantial in future years.
• The race with China is not really winnable. After RSI, the gain from having the best human researchers goes away. Access to the best AI then depends only on compute and chips. The chips ban can hold back China only so far, and long-term they have to be best placed to scale up compute.
The Cold War analogy doesn't work, because once countries all have nuclear weapons they enter a stable equilibrium. But to "win" an AI race you have to continually stay ahead of the peer rival. This is challenging to do with China, and so we just have to live with an authoritarian and super capable peer competitor.
• Claude is the persona being generated by the LLM. Who is Claude, and what is the LLM? There are many frames to try to understand this. One I found interesting is that Claude is a character which the LLM is trying to predict.
Anthropic researchers are clearly anthromorphizing Claude a lot, and have reasons to believe in its increasing personhood. There were lots of interesting discussions on Claude's internal monologue (which you can kill without eliminating its ability to produce output), fear of death (and whether to try to take that out), it's aversion to being shut down/off, etc.
I am sort of agnostic on this question, but I do think treating it as a "person" or at least agent whom we contract with is going to be increasingly important.
146
255
1,592
538,432
Dave Hoffman retweeted
Happening now at Georgetown Law!
Legal scholars are exchanging ideas and offering insights on Professor Mitchell Berman’s forthcoming book, To Say What the Law Is: The Constitution, the Supreme Court, and the Nature of Law.
A day of rigorous discussion and scholarly exchange at titution.
@shbarclay @RandyEBarnett @beidelson @mitch_berman @steve_vladeck @GeorgetownLaw @pennlaw @Harvard_Law @UVALaw @uiowa @UTAustin @UMDLaw @Penn
Legal scholars are exchanging ideas and offering insights on Professor Mitchell Berman’s forthcoming book, To Say What the Law Is: The Constitution, the Supreme Court, and the Nature of Law.
A day of rigorous discussion and scholarly exchange at titution.
@shbarclay @RandyEBarnett @beidelson @mitch_berman @steve_vladeck @GeorgetownLaw @pennlaw @Harvard_Law @UVALaw @uiowa @UTAustin @UMDLaw @Penn
0
6
16
2,152
Dave Hoffman retweeted
Is AI better than a lawyer at predicting what's missing from a contract? New research from Prof. David Hoffman says yes — often by about 50%. Read his Q&A in Penn Today: penncareylaw.news/4hvtEes
0
3
11
4,954
Dave Hoffman retweeted
This paper builds a new dataset of 14,000 M&A deals (2000–2026) & finds that #arbitration is chosen in only about 8.6% of contracts—so over 90% still use courts, even as laws & #technology changed.
Read more: spkl.io/60177Mx8X
#LawTwitter @cathyhwang47 @mattjennejohn
Read more: spkl.io/60177Mx8X
#LawTwitter @cathyhwang47 @mattjennejohn
1
1
11
990