Sam Lehman: What the Reinforcement Learning Renaissance Means for Decentralized AI
Episode publishedPodcast: The Delphi PodcastThe Delphi Podcast
Transcript preview
Preview only
You're now plugged in to the Delphi podcast. Hey, everyone. It's Tommy from Delphi Ventures. And today, I'm thrilled to have on Sam Lehman who, wrote one of the absolute best post on AI called the world's RL gym, what the RL renaissance means for, decentralized AI. Sam, how are you? How's it going?
Doing well. Really happy to be here. Thanks for having me on. Sam,
you're, so you're a partner at Symbolic Capital. You're a crazy good AI researcher. Tell us a bit about yourself before we dive into the post for everyone.
Yeah. So I've been at Symbolic for about three years now. Sort of start with the thought around inception, have been investing across Web three broadly over those three years. In a past life, I worked in TradFi. And before that, I actually, opened a craft brewery. So I've done a bunch of different things, but now it's symbolic. I've been very focused on the world of decentralized AI. It's a huge part of our funds thesis. We do precede seed investing mainly, and I've just really been enjoying exploring this world of Web three that's, for me, provided the most excitement that I've ever had inves
I mean, our our funds are definitely aligned on the thesis there and and the focus. I'm I'm excited to talk to you on this. Let let's start with the beginning of the post and see where the conversation takes us, but let's just open up with you give a really good brief history of AI and ML scaling. Right? The pretraining scaling laws, the importance of data in this training, scaling up these models. Maybe just, like, walk us through the genesis of the history of the AI scaling stuff. Yeah.
Definitely. So I think about this history in three phases. Phase one being focused on pretraining, phase two on inference time, test time, compute scaling, and then sort of this new, emergent phase, which is RL scaling, which is what the post is focused about. And, essentially, to sort of zoom maybe zoom out initially, my goal of the piece was to, you know, starting with this new RL renaissance that's upon us, trace my way back to figure out how we got here, what came before, and, specifically, like, you know, what lever levers were AI researchers pulling to scale and make more performant mode
Yeah. It's pretty crazy. So, basically, you found out that in the pre training phase where these AI labs are just throwing, you know, mass compute, basically, the the zero and one parameters in these models, there was too many parameters for the amount of data going in. So there there wasn't enough data to to effectively change those zeros and ones to make a performant model. Is that what happened?
Yeah. Exactly. They're basically by, caught up flat what we're called for the chinchilla scaling laws. They found that the optimal ratio of data to parameters was around roughly 20 tokens per parameter. But before they sort of codified these scaling laws, there were that ratio was off. There were too many parameters relevant to the data going in, and that sort of open researchers' eyes to this idea that we had to get more data going into these models.
That's awesome. So that's that's phase one, which you just covered. And then phase two is this inference time, you know, test time compute scaling, like, where the model is sort of thinking in in real time. Walk us through that a little bit more because, obviously, everyone's seen, like, ChatGPT think, but I don't think they have, like, the nuance that that you have with it. Yeah.
Keep exploring with a free account.
A free account preserves the short preview. Upgrade to Pro for the full transcript and episode chat.