The Future of Frontier Model Architectures with Walter Goodwin, Fractile Founder and CEO
Episode publishedPodcast: No Priors: Artificial Intellig...No Priors: Artificial Intelligence | Technology | Startups
Transcript preview
Preview only
Right now, we're trying to build a single chip. If you crack open an NVIDIA system, it has anywhere between kind of six and nine custom chips all built by NVIDIA to come together to build something that is really, really potent. So there's already a gap there. You know, it'd be great to be productive enough that we could start to build our own responses in that same kind of way. This idea of being able to have at any given moment a kind of rolling frontier of bets that you hope are deeply aligned in which you're ready to trigger the ramp of, this is a huge advantage if you can kind of build th
Hi, listeners. Welcome back to No Priors. Today, I'm here with Walter Goodwin, founder and CEO of full stack AI chip company, Fractile. We talk about what it means to be full stack as a company, the technical bets they're making, why he's so focused on memory bandwidth, his predictions for frontier model architectures in the future, and the structure of the chip market at the frontier today and tomorrow amongst NVIDIA, AMD, internal efforts, and this new class of accelerators. Welcome, Walter. Walter, thanks so much for doing this.
Very good to be here, Sarith.
You started this incredibly interesting company called Fractal. Can you give us a overview of what you guys do?
Fractal is a chip company. We build very, very fast inference chips for the world's largest models. This sort of bet on speed is something which we've had right from the outset. Started the company in summer of twenty twenty two. And I think at the time, we were starting to see two things. You know? One was obviously the arrival of foundation models trained on the Internet and generalizing across everything. And the other was, you know, a few kinda wise people saying, you know, we need to find a way to pour more compute into these models at test time. We need to find a way to take what we had
It's become a much more interesting chip landscape in the four years since you started the company. Where would you place yourself in the overall GPU and accelerator landscape for inference?
Yeah. It's a... There's a kind of extraordinary zoo now of options available. And if you look across this kind of entire space of kind of AI a six, one of the things that I think is very striking is there's a lot of relatively identikit chips out there. And this is kind of a structural... It's a kind of structural industry property where if you look especially across kind of hyperscaler efforts and, you know, Google kinda kicked this off with the TPU more than ten years ago, You see this paradigm where although there are a number of kind of proprietary internal chips, you know, Google with the
For people who don't work in the chip industry, what is the common abstraction or handoff between what their... What, you know, architecture focused players are doing and a player like Broadcom? Yeah. So I think the... If you're... Let's say
you're Google and you're designing the TPU, you have a number of people in your team who have a really strong understanding of the workloads that they're trying to accelerate. And so, you know, going back over ten years, this is a world that then sees you conceive of things like the Tensor Core, which is a designated circuit dreamt up by an architect that is exceptionally good at matrix multiplications because you have this insight that matrix multiplications are the overwhelming majority of the kind of flop count in these models that you're running. That's kind of the preserve of the architec
Keep exploring with a free account.
A free account preserves the short preview. Upgrade to Pro for the full transcript and episode chat.