Some thoughts on the Sutton interview
Date: Podcast: Dwarkesh Podcast
Key Insights
Bullish:
- $RL - Reinforcement learning systems are expected to be the foundation for successor AI systems after AGI is achieved, representing the next evolution in AI architecture
- $COMPUTE - AI development will increasingly rely on techniques that effectively and scalably leverage compute power, following the bitter lesson principle
Bearish:
- $LLM - Current LLM architectures are inefficient as they consume massive compute during deployment without learning, representing a fundamental architectural limitation