AI Could Take Over in 2029. Is It Already Too Late? | Ryan Greenblatt
Episode publishedPodcast: The MAD Podcast with Matt Turc...The MAD Podcast with Matt Turck
Transcript preview
Preview only
I think that the AI company CEOs understand that they're on the path of building wildly smarter than human systems, like super intelligent AI systems. They understand that we don't really have a, like, clear thought through plan for how to manage the risks from that. I wouldn't say superintelligence is bad. I would say it's dangerous. It seems pretty likely that you end up with AI takeover as a result of building superintelligence. And so you quickly get from the AIs that are fully automating r and d and AIs that are, quite superhuman at everything. It turned out that somewhere along this tran
Hi. I'm Matt Turk, partner at FirstMark. Welcome back to the MAD podcast. My guest today is Ryan Greenblatt, chief scientist at Redwood Research. Ryan is the researcher who first caught an AI faking its own alignment back in 2024, and today is one of the authors of AI 2040 plan a, the most detailed blueprint anyone has written for how The US and China can avoid a reckless race to superintelligence and an AI takeover that could start as early as 2029. As a fair warning, this is a fascinating episode that gets a little dark, and the last ten minutes or so are probably the darkest. The good news,
Yeah. So I think that the AI company CEOs understand that they're on the path of building wildly smarter than human systems, like super intelligent AI systems. They understand that we don't really have a, like, clear thought through plan for how to manage the risks from that, you know, how to ensure these AIs don't take over, how to ensure they remain under our control. And they understand that this would this is a decent chance of a unprecedented concentration of power, at least in the absence of of very active efforts by the people who end up with that power to redistribute it. I think that
Yes. And since you published AI 2040, there's a couple of things that happened that seem to be going in the general direction of of what you recommend. So, specifically, when I paused its Astra model of our safety, following the hugging face incident. And then 1,200 insiders published that letter a few weeks ago, including Dario asking the government for slowdown tools. Curious about what you make of it. Is that is that what you, are recommending that's starting to happen?
Yeah. I mean, I think these are these are good steps. I mean, I think that, like, specifically, they're, like, pacing the frontier letter. The idea is, like, we just need to have the tools to, you know, if we're in a position where we need to spend a bunch of effort on safety, which I think we may be in today, we we we're very likely to be in in the in the future when AIs are more capable. We need to have the ability to do that while also not making that so that the, you know, actors that are actually applying safety get overtaken by other actors. And so, basically, like I don't know. I mean,
And just to verbalize the question early in this conversation, what is so bad about superintelligence? Obviously, there's a lot of talk about scientific progress and curing cancer and your documents recommends pausing the rates towards the superintelligence. So why why is it so bad?
Yeah. I wouldn't say superintelligence is bad. I would say it's dangerous. Like, it's a very dangerous thing to create. So the most straightforward story for why, it's dangerous is that it seems pretty likely that on sort of a trajectory similar to the trajectory we seem to be finding ourselves on, you end up with AI takeover as a result of building superintelligence because the AIs are in a position where they can take over due to bein
Keep exploring with a free account.
A free account preserves the short preview. Upgrade to Pro for the full transcript and episode chat.
Go deeper with this episode
Generate a summary and deeper analysis from the existing transcript.