Some thoughts on the Sutton interview

Bullish:

• $RL - Reinforcement learning systems are expected to be the foundation for successor AI systems after AGI is achieved, representing the next evolution in AI architecture

  • “If the LLMs do get to AGI first, which is what I expect to happen, the successor systems that they build will almost certainly be based on Richard's vision.”

    Dwarkesh·0:00
• $COMPUTE - AI development will increasingly rely on techniques that effectively and scalably leverage compute power, following the bitter lesson principle

  • “The bitter lesson says that you want to come up with techniques which most effectively and scalably leverage compute.”

    Dwarkesh·0:00

Bearish:

• $LLM - Current LLM architectures are inefficient as they consume massive compute during deployment without learning, representing a fundamental architectural limitation

  • “Most of the compute that's spent on an LLM is used in running it during deployment. And yet it's not learning anything during this entire period.”

    Dwarkesh·0:00