#452 – Dario Amodei: Anthropic CEO on Claude, AGI & the Future of AI & Humanity
Episode publishedPodcast: Lex Fridman PodcastLex Fridman Podcast
Transcript preview
Preview only
The following is a conversation with Dario Amadeh, CEO of Anthropic, the company that created Claude that is currently and often at the top of most LLM benchmark leaderboards. On top of that, Dario and the anthropic team have been outspoken advocates for taking the topic of AI safety very seriously, and they have continued to publish a lot of fascinating AI research on this and other topics. I'm also joined afterwards by 2 other brilliant people from Anthropic. 1st, Amanda Askell, who is a researcher working on alignment and fine tuning of Claude, including the design of Claude's character and
So I can only describe it as it as, you know, as it relates to kind of my own experience, but I've been in the AI field for about, 10 years, And it was something I noticed very early on. So I first joined the AI world when I was, working at Baidu with Andrew Ng in late 2014, which is almost exactly 10 years ago now. And the first thing we worked on was speech recognition systems. And in those days, I think deep learning was a new thing. It had made lots of progress, but everyone was always saying we don't have the algorithms we need to succeed. You know, we we we we're we're not we're only mat
And, of course, the scaling here is bigger networks, bigger data, bigger compute. Yes.
All of those. In in particular, linear scaling up of bigger networks, bigger training times, and, more and and more data. So all of these things, almost like a chemical reaction. You know, you have 3 ingredients in the chemical reaction, and you need to linearly scale up the 3 ingredients. If you scale up one, not the others, you run out of the other reagents and the and the reaction stops. But if you scale up everything everything in series, then then the reaction can proceed. And,
of course, now that you have this kind of empirical science slash art, you can apply to other, more nuanced things like scaling laws applied to interpretability or scaling laws applied to post training or just seeing how does this thing scale. But the big scaling law, I guess, the underlying scaling hypothesis has to do with big networks, big data leads to intelligence.
Yeah. We've we've documented scaling laws in loss of domains other than language. Right? So, initially, the the paper we did that first showed it was in early 2020 where we first showed it for language. There was then some work late in 2020 where we showed the same thing for other modalities, like images, video, text to image, image to text, math. They all had the same pattern. And and you're right. Now there are other stages like post training or there are new types of reasoning models. And in in in all of those cases that we've measured, we see similar similar types of scaling laws.
A bit of a philosophical question, but what's your intuition about why bigger is better in terms of network size and data size? Why does it lead to more intelligent models?
So in my previous career as a as a biophysicist, so I did physics undergrad and then biophysics in in in in grad school. So I think back to what I know as a physicist, which is actually much less than what some of my colleagues at Anthropic have in terms of in terms of expertise in physics. There's this there's this concept called the one over f noise and one over x distributions, where we're often, you know, just just like if you add up a bunch of natural processes, you get a Gaussian. If you add up a bunch of kind of differently distributed natural processes, if you like if you like, take a
So there's the long tail, but also there's the height of the hierarchy of concepts that you'
Keep exploring with a free account.
A free account preserves the short preview. Upgrade to Pro for the full transcript and episode chat.