Satya Nadella — How Microsoft is preparing for AGI – Dwarkesh Podcast

Dwarkesh

Today, we are interviewing Satya Nadella. We being me and Dylan Patel, who is founder of Semi Analysis. Satya, welcome.

Satya

Thank you. It's great. Thanks for coming over at Atlanta.

Dwarkesh

Yeah. Thank you for giving us the tour of the new facility. It's been really cool to see. Absolutely. Satya and Scott Guthrie, Microsoft's EVP of Cloud and AI, give us a tour of their brand new Fairwater 2 Data center, the current most powerful in the world.

Scott

We try to 10x the training capacity every eighteen to twenty four months. And so this would be effectively a 10x increase, 10x from what GPD five was trained with. And so to put it in perspective, the number of optics, the network optics in this building is almost as much as all of Azure across all our data centers two and a half years ago.

Satya

It's kind of what? 5,000,000 network connections.

Dwarkesh

You've got all this bandwidth between different sites in a region and between the two regions. So is this like a big bet on scaling the future that you anticipate in the future there's gonna be some huge model that needs to require two whole different regions to train?

Satya

Fine. The goal is to be able to kind of aggregate these flops for a large training job and then put these things together across sites.

Dwarkesh

Right.

Satya

And the reality is you'll use it for training and then you'll use it for data gen, you'll use it for inference in all sort of ways. Not like it's going to be used only for one workload forever.

Satya Nadella — How Microsoft is preparing for AGI

Mentioned

Transcript