What's Missing Between LLMs and AGI - Vishal Misra & Martin Casado
Episode publishedPodcast: The a16z ShowThe a16z Show
Transcript preview
Preview only
Althropic makes great products. PlotCore is fantastic. Cowork is fantastic. But they are cranes of silicon doing matrix multiplication. They don't have consciousness. They don't have an inner monologue. You take an NLM and train it on pre 1916 or 1911 physics and see if it can come up with the theory of relativity. If it does, then we have AGI.
Just today, by the way, Dario allegedly said that you can't rule out that they're conscious.
You can rule out their conscious. I think. In common. To get to what is called AGI, I think there are two things that need to happen.
Five years ago, Vishal Misra got GPT-three to translate natural language into a domain specific language it had never seen before. It worked. He had no idea why. So he set out to build a mathematical model of how LLMs actually function. The result? A series of papers showing that transformers update their predictions in a precise, mathematically predictable way. In controlled experiments, the models match the theoretically correct answer almost perfectly. But pattern matching is not intelligence. LLMs learn correlation. They don't build models of cause and effect. To get to AGI, Misra argues,
Michelle, it's good to have
you in again. Great to be back.
This is one of my favorite topics, which is how do LMs actually work. Mhmm. And I think that and in my opinion, you've done kind of the best work on this modeling it out.
Thank you.
For those that did not see the original one, maybe it's probably worth doing just a quick background on kind of what led you to this point, and then we'll just go into the current work that you've been doing.
Keep exploring with a free account.
A free account preserves the short preview. Upgrade to Pro for the full transcript and episode chat.