Pinned Tweet
It's finally out! journals.sagepub.com/doi/10.1177/17… (OA)
New paper!
Got the best news on the weekend: "Why hypothesis testers should spend less time testing hypotheses", by @LeonidTiokhin, @peder_isager, @lakens, and myself, has been accepted at Perspectives on Psychological Science.
psyarxiv.com/vekpu
1
8
54
Anne Scheel retweeted
From today, Works in Progress is now in stock at Barnes & Nobles nationwide across America.
We're in 570 stores in 49 states*, including the flagship stores at Union Square & Fifth Avenue in NYC; Georgetown, DC; State Street, Chicago; and The Grove, Los Angeles.
We're also now in Hudson News outlets across most major US airports, and the B&Ns at San Francisco Airport.
If you tweet (or LinkedIn/Insta post) a photo of a copy of Works in Progress outside any of these stores, share the post with me or @WorksInProgMag via DM and we'll send you a free tote bag as a thank you. If you're a subscriber already, feel free to do it with own copy!
You can see a full map of Barnes & Noble stockists here, and enter your Zip code to find your nearest store.
*Sorry, Wyoming, we'll get to you!
worksinprogress.co/barnes-noble-l…
We're in 570 stores in 49 states*, including the flagship stores at Union Square & Fifth Avenue in NYC; Georgetown, DC; State Street, Chicago; and The Grove, Los Angeles.
We're also now in Hudson News outlets across most major US airports, and the B&Ns at San Francisco Airport.
If you tweet (or LinkedIn/Insta post) a photo of a copy of Works in Progress outside any of these stores, share the post with me or @WorksInProgMag via DM and we'll send you a free tote bag as a thank you. If you're a subscriber already, feel free to do it with own copy!
You can see a full map of Barnes & Noble stockists here, and enter your Zip code to find your nearest store.
*Sorry, Wyoming, we'll get to you!
worksinprogress.co/barnes-noble-l…
63
67
715
81,622
Anne Scheel retweeted
I’m so pleased to have Dan Wang in Works in Progress as a regular contributor. He is a rare analyst of geopolitics who appreciates both technology and culture, to the point where I expect his columns to cover food and music just as much as semiconductors and defence. I love his wry sense of humour and think he is one of the few people writing today who counts as a genuine “must read”.
We’re delighted to announce that @danwwang is making his debut as a contributor, with a column for this issue of @WorksInProgress.
worksinprogress.co/issue/sichuan-…
Few people understand the China–US contrast as well as Dan Wang, and fewer still write about it with so much talent. His book Breackneck has introduced the now famous meme of China as a “country of engineers” and United States as a “country of lawyers,” to explain the differences in building capacity and physical world stagnation we are seeing between the two polities.
In his new column for us, he takes up one specific thread in the long-running debate over how China and the US compare and argues that both owe part of their power to being polycentric.
One observation is that both the American and US capitals are not their biggest cities. Vienna, London, Seoul and Paris dwarf their countries’ second cities, by four to seven times. By contrast, Beijing isn’t China’s largest city, and DC isn’t even in the top five in the US. Does this make US and China lame, for having smol capitals? No! On the contrary, Dan thinks that this friction between political and economic centers is important and at least partially explains why both countries can combine dynamism with strong states.
But the two countries are polycentric in very different ways. China’s regions are culturally distinct and are shaped heavily by geography. While coastal provinces tend to look outward, mountain areas like Sichuan and Chongqing are characterized by evading the center’s edicts. Dan also points out some misconceptions: yes, China has central control, but it is weaker than it look. Local officials often ignore or quietly subvert senseless directives from Beijing, with enforcement thinning out quickly beyond the big cities.
The characteristic feature of American polycentrism is how well it sorts people into domains and “sends” them to specific cities: this is reminiscent of the point @patio11 has made around “The Sort.” In America, the smartest and most driven tend to cluster in a handful of cities, above all New York and San Francisco, with Washington and Boston are slightly behind them. More than this: each field has an obvious center: AI has its capital in SF, for policy and politics you have DC, for finance New York and for biotech Boston and more recently, SF.
Different cities in America also prize different things: Miami values looking good, Boston being intelligent and bookish, LA how famous you are and New York how rich you are. There is so much variety in terms of who you can choose to become in America. And the city you decide to live in might end up shaping who you are, moreso than in any other country!
1
8
174
7,571
Anne Scheel retweeted
New blog post, about a really cool new feature in Metacheck: Automatic computational reproducibility checks with Metacheck daniellakens.blogspot.com/2026/10/automa…
This should be of interest to reviewers, data managers, and anyone sharing data and code. Run the checks on our SURF hosted server!
This should be of interest to reviewers, data managers, and anyone sharing data and code. Run the checks on our SURF hosted server!
0
7
21
2,182
Anne Scheel retweeted
New record for Stack Overflow:
<1000 questions posted last month
<1000 questions posted last month
355
694
10,069
1,297,726
Anne Scheel retweeted
Some big personal news: I'm moving to join Works in Progress as executive editor, starting next month. I have been hoping for an opportunity to work with @bswud and @s8mb for more than a decade, and I'm incredibly excited to finally have one.
WIP is expanding on every front right now. We'll soon begin coverage of business and finance, as well as putting out more reported features. To that end, we'll be hiring more writers and reporters in the year to come. I'm planning to remain based in the US to help build out our presence here.
The print edition of WIP has existed for less than 12 months, but sometime in the next week or so we will reach our 10,000th paying print subscriber. It's a group that now includes people who've been at the highest echelons of government and who run the world's biggest public companies. We’re already on track to 3x that number next year. We'll be in store at Barnes & Noble, Hudson News and independent newsagents next week too.
I think WIP is providing something that people are craving from physical media right now: fewer, extremely well-crafted pieces in a beautifully designed form. Issues now feature an amazing group of columnists, including Tyler Cowen, Dan Wang and Stephen Kotkin. Each edition is better than the last.
You'll see us in more and more places next year too. In January, our new weekly documentary films will launch on YouTube. We're expanding into the events space, with a new series in North America. The sky is the limit from here on out.
WIP is expanding on every front right now. We'll soon begin coverage of business and finance, as well as putting out more reported features. To that end, we'll be hiring more writers and reporters in the year to come. I'm planning to remain based in the US to help build out our presence here.
The print edition of WIP has existed for less than 12 months, but sometime in the next week or so we will reach our 10,000th paying print subscriber. It's a group that now includes people who've been at the highest echelons of government and who run the world's biggest public companies. We’re already on track to 3x that number next year. We'll be in store at Barnes & Noble, Hudson News and independent newsagents next week too.
I think WIP is providing something that people are craving from physical media right now: fewer, extremely well-crafted pieces in a beautifully designed form. Issues now feature an amazing group of columnists, including Tyler Cowen, Dan Wang and Stephen Kotkin. Each edition is better than the last.
You'll see us in more and more places next year too. In January, our new weekly documentary films will launch on YouTube. We're expanding into the events space, with a new series in North America. The sky is the limit from here on out.
118
75
1,299
118,882
Anne Scheel retweeted
I’m very glad we have a cocktail column in Works in Progress. The rise of teetotalism in places like San Francisco is dangerous to civilization; this feels like a small stand against it. Hopefully we will add non-pretentious writing about wine at some point too.
The new issue of @WorksInProgMag both looks and feels fantastic (they are now using a matte paper instead of glossy).
Featured cocktail in this issue: Hanky Panky
18
24
642
33,316
Anne Scheel retweeted
All my work online for a guaranteed 100 years.
It's a new service being offered by WordPress. They tried it out on me. I think it's great.
Explained here: wordpress.com/blog/2025/02/1…
It's a new service being offered by WordPress. They tried it out on me. I think it's great.
Explained here: wordpress.com/blog/2025/02/1…
This is such a gift to the world. All of @stewartbrand’s work: writing, festival shenanigans, & 50,000(!) photos, dating back to the 50s: stewartbrand.org
(Where else could you find Marshall McLuhan, Brian Eno, and Marlon Brando in one place!?)
16
89
532
70,617
Anne Scheel retweeted
9 days left to submit an abstract to the 9th edition of the Perspectives on Scientific Error conference that takes place in Copenhagen, Denmark, 27–29 January 2027, errorsin.science. It's my favorite conference combining philosophy of science, metascience, and statistics.
0
1
7
1,225
Anne Scheel retweeted
We are excited to announce that we are going to broadcast the second day of Meta-Research Symposium 2026 (this Friday on October 9th) for those who were interested but unable to join in person.
JOIN: teams.microsoft.com/l/meetup-join/…
PROGRAM: docs.google.com/document/u/0/d…
JOIN: teams.microsoft.com/l/meetup-join/…
PROGRAM: docs.google.com/document/u/0/d…
0
2
13
2,102
Anne Scheel retweeted
The "Works in Progress-to-policy" pipeline is working.
When I stilled worked on biosecurity, I first learned about Debug, a Google project to reduce mosquito populations.
Results from their Singapore pilot are now out.
They crushed mosquito populations, reducing dengue infections by 70% in the process:
2
14
209
18,959
Anne Scheel retweeted
There is no good reason for mosquitoes to exist in wealthy countries today.
New technologies can get rid of the mosquito species that cause diseases like dengue, yellow fever, and malaria; and others can prevent them from carrying those diseases.
worksinprogress.co/issue/mosquito…
So why do mosquitoes still exist?
In a new article for Works in Progress, Maya Rosen explains how state incapacity and risk aversion led to over a decade of delays and billions of unnecessary bites in the United States.
Oxitec, for example, engineered an Aedes aegypti mosquito in such a way that its female offspring would die before adulthood. Through sustained releases, they showed they could reduce local mosquito populations by 80% or more, in field trials.
They sought approval from the USDA in 2010, which then passed it to the FDA, which passed it to the EPA. In 2020, the EPA allowed them to conduct limited field trials in the Florida Keys. Two years ago, their experimental permit expired, and they still need another review before it can be used routinely. Their proposed field trial in California was withdrawn after public opposition.
Another company, MosquitoMate, developed a technique involving Wolbachia, which also suppresses local mosquitoes. They then spent over fifteen years working through the system before finally receiving commercial registration nationwide in 2024.
It seems that, sometimes, the barrier to getting rid of diseases isn’t technology at all. It’s policy and political will.
New products shouldn’t get stuck in regulatory review for so long, especially if the delays are about basic questions like which agency has oversight over the product, rather than testing for safety.
Though some of those uncertainties were being addressed, the broader changes haven’t materialized. After Biden’s 2022 executive order, the agencies planned to update how their coordinated framework was implemented. But Trump rescinded the order last year.
Meanwhile, the EPA unit that reviews new mosquito control products is still a small office backlogged in reviewing a wide range of products.
Mosquitoes are a policy choice. We can get rid of them. Will we choose to?
worksinprogress.co/issue/mosquito…
New technologies can get rid of the mosquito species that cause diseases like dengue, yellow fever, and malaria; and others can prevent them from carrying those diseases.
worksinprogress.co/issue/mosquito…
So why do mosquitoes still exist?
In a new article for Works in Progress, Maya Rosen explains how state incapacity and risk aversion led to over a decade of delays and billions of unnecessary bites in the United States.
Oxitec, for example, engineered an Aedes aegypti mosquito in such a way that its female offspring would die before adulthood. Through sustained releases, they showed they could reduce local mosquito populations by 80% or more, in field trials.
They sought approval from the USDA in 2010, which then passed it to the FDA, which passed it to the EPA. In 2020, the EPA allowed them to conduct limited field trials in the Florida Keys. Two years ago, their experimental permit expired, and they still need another review before it can be used routinely. Their proposed field trial in California was withdrawn after public opposition.
Another company, MosquitoMate, developed a technique involving Wolbachia, which also suppresses local mosquitoes. They then spent over fifteen years working through the system before finally receiving commercial registration nationwide in 2024.
It seems that, sometimes, the barrier to getting rid of diseases isn’t technology at all. It’s policy and political will.
New products shouldn’t get stuck in regulatory review for so long, especially if the delays are about basic questions like which agency has oversight over the product, rather than testing for safety.
Though some of those uncertainties were being addressed, the broader changes haven’t materialized. After Biden’s 2022 executive order, the agencies planned to update how their coordinated framework was implemented. But Trump rescinded the order last year.
Meanwhile, the EPA unit that reviews new mosquito control products is still a small office backlogged in reviewing a wide range of products.
Mosquitoes are a policy choice. We can get rid of them. Will we choose to?
worksinprogress.co/issue/mosquito…
87
296
1,740
558,106
Anne Scheel retweeted
An interesting thing is happening to me at the moment. I have a standing bug bounty policy for my first-authored scientific work: I've committed to pay people who report bugs in my code, up to 750€ for severe errors. Since GPT Astra, I'm now getting several bug reports daily.
6
20
468
133,789
If you want X AND Y AND Z to be significant, you may even want to *increase* alpha for the individual comparisons to not lose power (and keep your overall alpha at the nominal level).
Which you should preregister of course, because people absolutely won’t buy it if you don’t.
Which you should preregister of course, because people absolutely won’t buy it if you don’t.
I am afraid @stuartbuck1 is getting multiple comparisons wrong. If you want outcome X AND Y AND Z to be significant (as in the example) you do not correct so the problem never happens. You correct if you want X OR Y OR Z to show an effect.
0
0
3
414
Anne Scheel retweeted
We are on the cusp of a wave of new therapies for some of the worst diseases. But the world won’t benefit unless the US fixes its drug regulatory system. My new essay for @nytimes, on how slow clinical trials are now the biggest obstacle to curing cancer.
nytimes.com/2026/09/04/opi…
- I interviewed dozens of researchers, especially oncologists at leading U.S. centers. A striking consensus emerged: science is no longer the main bottleneck to new cancer drugs. It is our ability to test discoveries in patients through clinical trials.
- The cost of starting a Phase 1 trial in America has roughly doubled over the past decade. As a result, companies increasingly take early trials abroad: Australia’s Phase 1 trial volume has nearly doubled in a decade, driven mainly by U.S. companies.
- Unfortunately, the underlying incentives are badly asymmetric: Institutions can be blamed for harms caused by moving too fast, but almost no one is blamed when patients deteriorate during avoidable delays. One doctor called the emerging system “ritualized safety over actual risk assessment.” Or as, @DavidHongMD put it: "We often forget that the biggest risk is the cancer itself."
- This problem is becoming more urgent because medicine itself is changing. Sequencing, biological engineering and A.I. make increasingly personalized therapies possible. But our regulatory system was mostly built for standardized drugs tested in large populations.
- @sytse, the co-founder of GitLab, shows what personalized medicine can achieve: after relapsed osteosarcoma and being told there were no options left, he pursued a highly individualized approach and has now been cancer-free for a year. But doing so required extraordinary resources and regulatory expertise.
- Pierce Ogden’s father was less lucky. After molecular analysis identified a drug that might target his glioblastoma, the manufacturer agreed to provide it. But administrative barriers delayed access until it was too late. “My dad was ready to try anything,” Pierce told me. “But the system is paternalistic.”
- The A.I. revolution is making this bottleneck more important, not less. A.I. relies on relevant data. Information from early-stage trials could compound with A.I. tools to achieve truly revolutionary medicines. Without the data, this is far less likely to happen.
- Another important shift is that innovation increasingly comes from academic labs and small biotech companies rather than Big Pharma. These small companies find it far harder to unable to absorb delays and regulatory barriers.
- Apart from cancer, China is the biggest winner from America's outdated medical regulations. China has a much faster trial system, with testing often starting a full year earlier. This allows Chinese pharmaceutical companies to experiment and improve medicines while American companies play with mice. Today, half of all drugs licensed by major pharmaceutical companies originate there, up from less than 5 percent only a decade ago.
- But we don't need to copy China. The best model is Australia: lots of on-site scientific and ethics reviews, and requirements that are proportionate to small, early-stage trials. Phase 1 studies there begin roughly 6–12 months faster, without any notable increases in adverse safety events.
- Operation TrialBlazer, a 2026 HHS initiative is a good start in this direction, but we need legislative action by Congress to truly make Phase I trials faster and more efficient!
I want to thank everyone who helped me with this article: everyone I interviewed and the amazing editors at the Times. This is the result of a months long journey of extensive interviews and research.
Special thanks go to those who came on the record. One of the features of the system is an atmosphere of fear, where practitioners are afraid to publicly come out and explain these issues. So anyone who does is a hero in my book!
nytimes.com/2026/09/04/opi…
- I interviewed dozens of researchers, especially oncologists at leading U.S. centers. A striking consensus emerged: science is no longer the main bottleneck to new cancer drugs. It is our ability to test discoveries in patients through clinical trials.
- The cost of starting a Phase 1 trial in America has roughly doubled over the past decade. As a result, companies increasingly take early trials abroad: Australia’s Phase 1 trial volume has nearly doubled in a decade, driven mainly by U.S. companies.
- Unfortunately, the underlying incentives are badly asymmetric: Institutions can be blamed for harms caused by moving too fast, but almost no one is blamed when patients deteriorate during avoidable delays. One doctor called the emerging system “ritualized safety over actual risk assessment.” Or as, @DavidHongMD put it: "We often forget that the biggest risk is the cancer itself."
- This problem is becoming more urgent because medicine itself is changing. Sequencing, biological engineering and A.I. make increasingly personalized therapies possible. But our regulatory system was mostly built for standardized drugs tested in large populations.
- @sytse, the co-founder of GitLab, shows what personalized medicine can achieve: after relapsed osteosarcoma and being told there were no options left, he pursued a highly individualized approach and has now been cancer-free for a year. But doing so required extraordinary resources and regulatory expertise.
- Pierce Ogden’s father was less lucky. After molecular analysis identified a drug that might target his glioblastoma, the manufacturer agreed to provide it. But administrative barriers delayed access until it was too late. “My dad was ready to try anything,” Pierce told me. “But the system is paternalistic.”
- The A.I. revolution is making this bottleneck more important, not less. A.I. relies on relevant data. Information from early-stage trials could compound with A.I. tools to achieve truly revolutionary medicines. Without the data, this is far less likely to happen.
- Another important shift is that innovation increasingly comes from academic labs and small biotech companies rather than Big Pharma. These small companies find it far harder to unable to absorb delays and regulatory barriers.
- Apart from cancer, China is the biggest winner from America's outdated medical regulations. China has a much faster trial system, with testing often starting a full year earlier. This allows Chinese pharmaceutical companies to experiment and improve medicines while American companies play with mice. Today, half of all drugs licensed by major pharmaceutical companies originate there, up from less than 5 percent only a decade ago.
- But we don't need to copy China. The best model is Australia: lots of on-site scientific and ethics reviews, and requirements that are proportionate to small, early-stage trials. Phase 1 studies there begin roughly 6–12 months faster, without any notable increases in adverse safety events.
- Operation TrialBlazer, a 2026 HHS initiative is a good start in this direction, but we need legislative action by Congress to truly make Phase I trials faster and more efficient!
I want to thank everyone who helped me with this article: everyone I interviewed and the amazing editors at the Times. This is the result of a months long journey of extensive interviews and research.
Special thanks go to those who came on the record. One of the features of the system is an atmosphere of fear, where practitioners are afraid to publicly come out and explain these issues. So anyone who does is a hero in my book!
82
415
1,608
557,497
Anne Scheel retweeted
Throwing rubbish in a landfill is (usually) better for the environment than recycling it. With few exceptions, recycling ends up emitting more CO2 and costing more money, while landfills are cheap and secure.
- Recycling mid-tier plastics (like plastic bottles or butter tubs) costs thousands of dollars per ton of CO2 saved. It's cheaper to do direct air CO2 capture than that! And there is no shortage of plastic, nor any problem with sequestering it in the ground once we're done with it.
- Modern landfills are extraordinarily secure and are *much* more than just big piles of trash. They capture 75%–95% of methane gas. Groundwater contamination is doesn't really happen with landfills built since 1990. A study of 31 landfills built since 1988 in NY found no adverse effect at all on local groundwater quality.
- The crusade against single-use plastics has driven us to products that use far more materials and energy and would need to be used an absurd number of times to actually be an improvement. A steel straw needs to be used 150 times to be worth using. A ceramic mug needs to be used 1,000 times to beat a polystyrene cup. A cotton bag needs to be reused 173 times to beat single-use.
- Some things ARE worth recycling. Recycling metal saves a huge amount of energy and material – recycling aluminium saves 95% of the energy spent on making fresh stuff. So we do a lot of it already: about 30% of steel worldwide is recycled.
Modern, well-run landfills cause minimal environmental damage. We have plenty of space for them – less than 0.01% of the US is used for landfill and only about 0.02% of Germany. We can even build on top of old landfills, so the land area is not spent permanently.
New in @WorksInProgMag, @robertwiblin and @chalmermagne on why it's usually best to just bury your trash. worksinprogress.co/issue/just-bur…
- Recycling mid-tier plastics (like plastic bottles or butter tubs) costs thousands of dollars per ton of CO2 saved. It's cheaper to do direct air CO2 capture than that! And there is no shortage of plastic, nor any problem with sequestering it in the ground once we're done with it.
- Modern landfills are extraordinarily secure and are *much* more than just big piles of trash. They capture 75%–95% of methane gas. Groundwater contamination is doesn't really happen with landfills built since 1990. A study of 31 landfills built since 1988 in NY found no adverse effect at all on local groundwater quality.
- The crusade against single-use plastics has driven us to products that use far more materials and energy and would need to be used an absurd number of times to actually be an improvement. A steel straw needs to be used 150 times to be worth using. A ceramic mug needs to be used 1,000 times to beat a polystyrene cup. A cotton bag needs to be reused 173 times to beat single-use.
- Some things ARE worth recycling. Recycling metal saves a huge amount of energy and material – recycling aluminium saves 95% of the energy spent on making fresh stuff. So we do a lot of it already: about 30% of steel worldwide is recycled.
Modern, well-run landfills cause minimal environmental damage. We have plenty of space for them – less than 0.01% of the US is used for landfill and only about 0.02% of Germany. We can even build on top of old landfills, so the land area is not spent permanently.
New in @WorksInProgMag, @robertwiblin and @chalmermagne on why it's usually best to just bury your trash. worksinprogress.co/issue/just-bur…
268
789
5,478
1,619,500
Anne Scheel retweeted
Get clued into the news months in advance by subscribing for free to my Substack 🪩
5
10
92
13,942
Anne Scheel retweeted
1. I love the idea of curing most (or all?) diseases. I just wrote an article about it worksinprogress.news/p/future-of-me… and gave a TED talk about it too! ted.com/talks/saloni_d…
We should be a lot more ambitious with medical innovation, and the history of science makes me more, not less, optimistic about what’s possible. Many diseases we once thought were untreatable turned out not to be.
2. But what’s possible isn’t the same as what’s going to happen, and it’s very different from claiming it’s possible within a timeline of 10 years.
Even if we grant exponential improvement in our technological capabilities, we still wouldn’t be able to cure most diseases within ten years, because technology is just one part of the equation.
One clear illustration of this is Eroom’s law, where the cost of drug development has risen despite massive technological advancement.
3. In our episode ‘Will AI solve medicine?’ (harddrugs.worksinprogress.co/episodes/will-…) Jacob Trefethen and I talk about how funding, institutions, incentives, better measurement, and a lack of data mean that we’ll still have massive gaps in solving diseases despite progress in AI.
These problems fall most heavily on neglected diseases, like rare and tropical diseases, where funding is often the barrier to progress. But they also affect diseases whose clinical trials are long and onerous, disease progression itself is slow, or poorly measured.
4. All this means that, if we want to speed up medical innovation and scale it up worldwide, we should also be unblocking the other barriers to medical progress with clinical trial reform, better data collection, better measurement instruments, better funding models, and more. (See our ‘Clinical Trials Abundance blog’ for some ideas clinicaltrialsabundance.blog)
It’s going to take a while before we solve these to the point where we can cure most diseases!
5. I thought the policy proposals in ‘Policy on the AI Exponential’ relating to drug development were quite vague and too brief to be useful.
But even if they were all implemented ideally, we would still be lacking cures for diseases for which financing or clinical trial infrastructure are the barrier, or where we’d need entirely new drug modalities to reach targets safely.
FWIW, I’m fairly skeptical of the use of ‘synthetic control arms’ if we actually want reliably effective medicines, not just faster clinical trials. But platform trials, which include shared control groups that many drugs are tested against, probably get at the intention.
6. I’m very skeptical of the idea that “many of the steps in the clinical process that previously required expensive and slow experiments may soon be done via AI simulation or analysis” ... but maybe this depends on one’s definition of ‘many’.
Human biology is extremely complicated and interactions between drugs and molecules in our body are very hard to predict, in part because of the number of combinations and biological states, and the level of measurement that’s possible in the lab; the way we get around this is through experimental data collection, i.e. clinical trials, but for some diseases, progression is very slow, so we’re not going to be able to collect that amount of data in time, even with perfect measurement.
Unfortunately public health datasets typically don’t include much drug-human body data, with the exception of, e.g., prescription datasets, which would be somewhat useful for analyzing historical drugs.
Getting pharma companies to pool sufficient individual participant data from clinical trials to improve these predictions, though, that would be exciting.
7. I sometimes worry about the deluge of AI ‘slop’ science that’s going to enter the academic literature.
We already needed to reform peer review, catch scientific fraud, improve data quality and rigour before AI (see e.g. my articles ‘Real peer review has never been tried’ worksinprogress.co/issue/real-pee… and ‘The speed of science’ worksinprogress.co/issue/the-spee…), but now it seems like science reform is even more important.
I could imagine a Pangram-like model for AI-written science papers, but that seems like it merely catches the tip of the iceberg of how deep the slop might go, if it involves unpublished data, for example. If AI use increases exponentially, we might end up seeing an AI science slop tsunami, which could affect predictions based on the literature. What’s the solution to this?
We should be a lot more ambitious with medical innovation, and the history of science makes me more, not less, optimistic about what’s possible. Many diseases we once thought were untreatable turned out not to be.
2. But what’s possible isn’t the same as what’s going to happen, and it’s very different from claiming it’s possible within a timeline of 10 years.
Even if we grant exponential improvement in our technological capabilities, we still wouldn’t be able to cure most diseases within ten years, because technology is just one part of the equation.
One clear illustration of this is Eroom’s law, where the cost of drug development has risen despite massive technological advancement.
3. In our episode ‘Will AI solve medicine?’ (harddrugs.worksinprogress.co/episodes/will-…) Jacob Trefethen and I talk about how funding, institutions, incentives, better measurement, and a lack of data mean that we’ll still have massive gaps in solving diseases despite progress in AI.
These problems fall most heavily on neglected diseases, like rare and tropical diseases, where funding is often the barrier to progress. But they also affect diseases whose clinical trials are long and onerous, disease progression itself is slow, or poorly measured.
4. All this means that, if we want to speed up medical innovation and scale it up worldwide, we should also be unblocking the other barriers to medical progress with clinical trial reform, better data collection, better measurement instruments, better funding models, and more. (See our ‘Clinical Trials Abundance blog’ for some ideas clinicaltrialsabundance.blog)
It’s going to take a while before we solve these to the point where we can cure most diseases!
5. I thought the policy proposals in ‘Policy on the AI Exponential’ relating to drug development were quite vague and too brief to be useful.
But even if they were all implemented ideally, we would still be lacking cures for diseases for which financing or clinical trial infrastructure are the barrier, or where we’d need entirely new drug modalities to reach targets safely.
FWIW, I’m fairly skeptical of the use of ‘synthetic control arms’ if we actually want reliably effective medicines, not just faster clinical trials. But platform trials, which include shared control groups that many drugs are tested against, probably get at the intention.
6. I’m very skeptical of the idea that “many of the steps in the clinical process that previously required expensive and slow experiments may soon be done via AI simulation or analysis” ... but maybe this depends on one’s definition of ‘many’.
Human biology is extremely complicated and interactions between drugs and molecules in our body are very hard to predict, in part because of the number of combinations and biological states, and the level of measurement that’s possible in the lab; the way we get around this is through experimental data collection, i.e. clinical trials, but for some diseases, progression is very slow, so we’re not going to be able to collect that amount of data in time, even with perfect measurement.
Unfortunately public health datasets typically don’t include much drug-human body data, with the exception of, e.g., prescription datasets, which would be somewhat useful for analyzing historical drugs.
Getting pharma companies to pool sufficient individual participant data from clinical trials to improve these predictions, though, that would be exciting.
7. I sometimes worry about the deluge of AI ‘slop’ science that’s going to enter the academic literature.
We already needed to reform peer review, catch scientific fraud, improve data quality and rigour before AI (see e.g. my articles ‘Real peer review has never been tried’ worksinprogress.co/issue/real-pee… and ‘The speed of science’ worksinprogress.co/issue/the-spee…), but now it seems like science reform is even more important.
I could imagine a Pangram-like model for AI-written science papers, but that seems like it merely catches the tip of the iceberg of how deep the slop might go, if it involves unpublished data, for example. If AI use increases exponentially, we might end up seeing an AI science slop tsunami, which could affect predictions based on the literature. What’s the solution to this?
2/2 Second, on the messaging around AI. I do not agree that my messaging has been disproportionately negative. In fact it has been about equally balanced between risks and benefits: I’ve written one major essay about each, and even in interviews where I discuss the risks, I make sure to frequently mention the incredible benefits as well as proposing possible solutions to the risks (short clips from my interviews that end up on social media tend to be disproportionately negative, as that gets clicks). In fact, I wrote Machines of Loving Grace because I didn’t feel the AI industry was painting an inspiring enough picture of how the technology could radically transform the world for the better. The bulk of the essay is devoted to refuting skepticism of AI’s potential in health and biology, and showing why I think it will actually be possible to cure most human disease in ~5-10 years, as crazy as it may sound to ordinary people and frankly to biologists as well (I used to be one!). And, if you read my most recent essay (Policy on the AI Exponential), I discuss concrete proposals for how to streamline the FDA process to make sure the deluge of AI-accelerated drugs isn’t slowed down by the regulatory process. I feel the urgency here: I lost my father to Hepatitis C only a few years before the development of direct-acting antivirals (sofosbuvir), which cure 95% of patients and probably would have cured him.
I do agree that the public has a negative view of AI (and that this is a big problem), but I don’t think it is primarily caused by me or any other AI leader warning about AI’s risks. I think it is fundamentally a crisis of trust. I think that ordinary people don’t trust companies, governments, or the tech industry and always suspect that we are cooking up some new way to screw them over. The causes of this go back decades and AI is just the latest iteration of it. I don’t think that a glitzy marketing campaign with a positive spin (which some have advocated that Anthropic do) is the way to win back that trust — at this point, saying that AI will cure cancer is more a cliche than it is inspiring, and most people think it is deceptive. The thing that will work is *actually curing cancer*. I think by far the most accurate criticism of AI companies including Anthropic is that we haven’t yet delivered on our big promises to benefit the world. That is totally on us, and I think it’s the criticism you should be making, instead of all this stuff about messaging and marketing.
We are however doing our best to fix this: Anthropic is ramping up its efforts very quickly in biology and medicine, and we hope to have incredible results in the coming years and some early glimmers in the coming months. When we’ve actually accomplished something real, the whole world will hear about it, as loudly as possible, you have my word on that. But until then I don’t want to make empty promises, and in the meantime I feel compelled to speak honestly about the very real risks of AI and how to address them. Honesty is the right thing on the merits, and in terms of public credibility and trust it is no worse than, and may in fact be better than, an approach that ignores or distracts from risks which people instinctively understand are real.
9
32
144
15,700
Anne Scheel retweeted
Saloni’s talk gave me goosebumps. You should watch it.
It's finally here! I gave a TED talk in April in Vancouver, and it was released online today.
You can watch it here:
ted.com/talks/saloni_d…
I've also adapted it into an article, which was published last week: worksinprogress.co/issue/future-o…
1
6
91
31,624