The next big breakthrough will be AIs learning on the job
Episode publishedPodcast: Dwarkesh PodcastDwarkesh Podcast
Transcript preview
Preview only
So here's the big research bet that all the labs are making. They think that if we train AIs to accomplish millions of verifiable tasks across thousands of diverse RL environments, then we will have basically built AGI. Because this kind of training will have created a kind of problem solving agent. The kind of thing that can make progress on opening the task for weeks on end in the face of errors and mistakes and ambiguity. And the people who are optimistic about this vision will say that all these things that we talk about as the fundamental deficits in the current training paradigm, for exa
There's two things. There's the context length you train at, and there's a context length that you serve at. If you train at a small context length and then try to serve at a long context length,
like maybe you get these degradations. Now maybe I'm reading too much into this, but it seems like he's saying that short horizon RL training doesn't necessarily generalize to long horizon RL performance. And if you can't generalize from short horizon to long horizon, then how are agents supposed to generalize from getting trained at a bunch of white collar tasks to, say, having the ability to be dropped in the real world and build a business from scratch as well as Sam Walton? And even if after enough in context experience, the AIs can become like Henry Ford or Albert Einstein, All that would
You give it a work review. And if you give it
a thumbs up, the base model distills everything that the AI learned during the session. And it may use OPSD, it may use dreaming, it may use some other technique that we aren't even aware of, or it'll use a combination of all of the above. In this way, the gamut of AI skills and knowledge and capabilities can expand far beyond the verifiable domains that the model was originally trained against before it was deployed. Just as pre training created a base intelligence that was smart enough to become a competent agent with enough ROVR on top. So ROVR has created an agent that is competent enough
Keep exploring with a free account.
A free account preserves the short preview. Upgrade to Pro for the full transcript and episode chat.