Crazy, true story: Minnesota offered an expert declaration on AI and “misinformation” to oppose our motion to enjoin their unconstitutional law.
His declaration included fake citations hallucinated by AI!
🧵 1/x
His declaration included fake citations hallucinated by AI!
🧵 1/x
Nov 19, 2024 · 7:01 PM UTC
21
157
751
315,196
Minnesota AG @keithellison offered the expert declaration in support of the state’s ban on political “deepfakes,” which we argue infringes core political speech. hlli.org/kohls-v-elliso… 2/
2
10
138
9,210
@keithellison His report claims that “One study found that even when individuals are informed about the existence of deepfakes, they may still struggle to distinguish between real and manipulated content.”
I guess the struggle is real because this study does not exist! 3/
I guess the struggle is real because this study does not exist! 3/
4
24
205
8,456
@keithellison The doi link is broken. No content with this title exists online. The journal exists, but those pages point to unrelated articles. The authors seem to be in an uncanny valley of academics who publish about deepfakes, but their names are slightly wrong. 4/
2
11
163
7,529
@keithellison This was really weird for me to discover; I wasn’t sure of myself. Maybe the title was slightly wrong. But I couldn’t find anything even close to the article’s description, so we filed a motion to exclude the testimony. hlli.org/wp-content/upl… 5/
3
7
123
8,247
As far as we can tell, this is the first time that an expert has cited hallucinated content in court. Eugene at @VolokhC has followed AI-generated content in the courts closely. No one else seems to have called out hallucinated expert citations before. 6/
4
14
170
12,871
@VolokhC Volokh also discovered that there’s a second hallucinated citation in the declaration. The author & journal are real, but this article does not exist and is not at the cited location. Some puckish AI invented it! reason.com/volokh/2024/11… 7/
2
12
135
9,522
@VolokhC (The unplanned additional motion to exclude the expert is why I was working to the last minute on Friday night.) x.com/FrankBednarz/s… 8/
Every time I'm working up to a midnight deadline as a lawyer, file, then look outside and people are like coming home from parties, uninterested in the feat.
1
4
89
7,643
Something to consider when the director of the Stanford Internet Observatory (@stanfordio) opines that fact-checks don't work & censorial laws are “critical in preserving the integrity of democratic institutions” from AI deepfakes.
Did an AI write this sentence too? 9/fin
Did an AI write this sentence too? 9/fin
1
8
115
4,641
The gist of his report is that counterspeech no longer works (and therefore government censorship is necessary). I think that's incorrect, and we hopefully prove our point by calling out AI misinformation to the court. x.com/aeschulman/sta…
@fedjudges My favorite part is how we have now performatively refuted his expert opinion. Counter speech does work! Don't fuck with @frankbednarz!
1
9
112
5,898
Interesting debate in @VolokhC comments about whether the fake citations might have been intentionally illustrative about the problems of deepfakes.
The perjury jurat makes me think this unlikely, but we'll see how the state responds.
The perjury jurat makes me think this unlikely, but we'll see how the state responds.
3
5
99
5,233
Reporting on the story: @_cingraham for the @MNReformer. Still no comment from the expert, who told Prof. Volokh that he would have a statement in a few days. minnesotareformer.com/2024/11/20/mis…
3
2
23
1,015
Protip: LLMs are eager to please, sometimes make stuff up. x.com/tedfrank/statu…
Always ask ChatGPT the follow up question. It gave me 2 other hallucinated cases & got the holdings of the 3 real ones wrong. I don’t see how any attorney could use it and not realize that it’s entirely worthless for litigation except maybe helping to reword an awkward sentence.
1
1
18
1,115
Update: still no comment from the expert, but the AG will “reply in court shortly” according to reporting by @Law360 that quotes co-counsel @james_vf_dickey at @UMLawCenter and me (at @HamLincLaw). law360.com/articles/22643…
1
1
15
739
@james_vf_dickey @HamLincLaw Conservatives pounce... on fabricated citations in a declaration under oath. sfgate.com/tech/article/s…
3
2
29
765
Update thread with link to new filing.
cc: folks who previously wrote on this: @VolokhC, @_cingraham, @hkonnath, @ToddFeathers, @stephencouncil, @instapundit, @CSJeffrey. x.com/FrankBednarz/s…
cc: folks who previously wrote on this: @VolokhC, @_cingraham, @hkonnath, @ToddFeathers, @stephencouncil, @instapundit, @CSJeffrey. x.com/FrankBednarz/s…
🚨🚨🚨 Update 🚨🚨🚨
Jeff Hancock, the $600/hr “AI and misinformation” that Minnesota's AG @keithellison hired to defend criminalizing political deepfakes admits his declaration includes fake journal articles hallucinated by ChatGPT.
1
0
11
851
@FrankBednarz Hey I'm a reporter with the Minnesota Reformer, I sent you an email about this earlier today. Any new information about what happened here? Wild story!
1
0
6
551
@FrankBednarz This is an amazing story. 😂This Minnesotan thanks you.
1
0
10
1,273
@AaronK_MN You're welcome. And a BYU grad! Tough loss this weekend; I saw that game.
1
0
3
1,154
@FrankBednarz @politicalmath Seeing this more and more. Had it in a case as well.
1
0
7
1,540
@TXLAW24 @politicalmath With an expert, or attorney? If an expert, would you mind telling me the case?
1
0
3
1,324
@FrankBednarz great piece. I was just dealing with an "expert" where I found similar issues in just-off citations - I wonder if those too were AI hallucinations.
1
0
6
1,037
@AshmoreHardy The thing that seems to characterize these hallucinated citations is they use real journal articles, and sometimes even real relevant academics, but the studies they imagine don't exist in the journal when you look them up. amjmed.com/article/S0002-…
0
0
9
924
@FrankBednarz Motion for sanctions seems appropriate.
The reason for citations is to facilitate finding supporting materials. An erroneous citation already adds time and thus cost. However, proving the negative proposition that a cited material simply does not exist is even more burdensome.
The reason for citations is to facilitate finding supporting materials. An erroneous citation already adds time and thus cost. However, proving the negative proposition that a cited material simply does not exist is even more burdensome.
0
1
3
336
@FrankBednarz @politicalmath Serve the Computer! The Computer is your friend!
Friend Computer wants you to be happy. If you are not happy, you may be recycled as reactor shielding.
Friend Computer wants you to be happy. If you are not happy, you may be recycled as reactor shielding.
0
0
3
384