🐍 Newsletters
AI Snake Oil
39 min read
Open-world evaluations for measuring frontier AI capabilities
Introducing CRUX, a new project for evaluating AI on long, messy tasks
AI Snake Oil
20 articles
🐍 Newsletters
AI Snake Oil
10 min read
New Paper: Towards a science of AI agent reliability
Quantifying the capability-reliability gap
🐍 Newsletters
AI Snake Oil
36 min read
AI Won’t Automatically Make Legal Services Cheaper
Applying the AI as Normal Technology framework to legal services
🐍 Newsletters
AI Snake Oil
13 min read
Fact checking Moravec's paradox
This famous aphorism is neither true nor useful
🐍 Newsletters
AI Snake Oil
A guide to understanding AI as normal technology
And a big change for this newsletter
🐍 Newsletters
AI Snake Oil
AGI is not a milestone
There is no capability threshold that will lead to sudden impacts
🐍 Newsletters
AI Snake Oil
AI as Normal Technology
A new paper that we will expand into our next book
🐍 Newsletters
AI Snake Oil
Is AI progress slowing down?
Making sense of recent technology trends and claims
🐍 Newsletters
AI Snake Oil
We Looked at 78 Election Deepfakes. Political Misinformation is not an AI Problem.
Technology Isn’t the Problem—or the Solution.
🐍 Newsletters
AI Snake Oil
Does the UK’s liver transplant matching algorithm systematically exclude younger patients?
Seemingly minor technical decisions can have life-or-death effects
🐍 Newsletters
AI Snake Oil
FAQ about the book and our writing process
What's in the book and how we wrote it
🐍 Newsletters
AI Snake Oil
Can AI automate computational reproducibility?
A new benchmark to measure the impact of AI on improving science
🐍 Newsletters
AI Snake Oil
Start reading the AI Snake Oil book online
The book was published September 2024
🐍 Newsletters
AI Snake Oil
AI companies are pivoting from creating gods to building products. Good.
Turning models into products runs into five challenges
🐍 Newsletters
AI Snake Oil
AI existential risk probabilities are too unreliable to inform policy
How speculation gets laundered through pseudo-quantification
🐍 Newsletters
AI Snake Oil
New paper: AI agents that matter
Rethinking AI agent benchmarking and evaluation
🐍 Newsletters
AI Snake Oil
Scientists should use AI as a tool, not an oracle
How AI hype leads to flawed research that fuels more hype
🐍 Newsletters
AI Snake Oil
AI leaderboards are no longer useful. It's time to switch to Pareto curves.
What spending $2,000 can tell us about evaluating AI agents