 |
Earley AI PodcastAuthor: Seth Earley
In this podcast hosts Seth Earley invites a broad array of thought leaders and practitioners to talk about what's possible in artificial intelligence as well as what is practical in the space as we move toward a world where AI is embedded in all aspects of our personal and professional lives. They explore what's emerging in technology, data science, and enterprise applications for artificial intelligence and machine learning and how to get from early-stage AI projects to fully mature applications. Seth is founder & CEO of Earley Information Science and the award-winning author of "The AI Powered Enterprise." Language: en-us Genres: Business, Technology Contact email: Get it Feed URL: Get it iTunes ID: Get it |
Listen Now...
Earley AI Podcast Episode 89: Memory, Power, and the Hidden Constraints of AI Infrastructure
Episode 89
Wednesday, 6 May, 2026
Guest: Steven Woo, Fellow and Distinguished Inventor at RambusHost: Seth Earley, CEO at Earley Information SciencePublished on: May 5, 2026In this episode, Seth Earley speaks with Steven Woo, Fellow and Distinguished Inventor at Rambus, where he has spent over 30 years at the frontier of memory technology. They explore why memory - not compute - is the binding constraint on AI performance today, how moving data between chips consumes more than half of all power in a high-end AI processor, and what the rise of agentic AI means for infrastructure planning. Steven shares a rare long-view perspective on the innovation curve for memory technology, the supply-demand dynamics driving prices higher, and the questions enterprise leaders should be asking before signing their next infrastructure contract.Key Takeaways:Memory, not compute, is the primary bottleneck limiting AI performance - and the gap between processor speed and memory speed is widening, not closing.Over 50 percent of the power consumed by high-end AI processors is spent simply moving data on and off the chip, not performing computation.Stacking memory components closer together can reduce energy costs dramatically but introduces new challenges around heat dissipation and power delivery.Training and inference have very different memory profiles - understanding both is essential for organizations architecting AI infrastructure at scale.Agentic AI compounds the memory challenge significantly, because one user can spin up multiple agents that each spawn further agents, multiplying context and capacity demands.Memory prices have risen sharply due to supply-demand imbalance - organizations are now signing long-term supply agreements to lock in capacity, just as they do for power.The most important question enterprise leaders can ask their infrastructure providers is how much experience and demonstrated reliability they have - downtime during model training can be catastrophic.Insightful Quotes:"Memory has become a big bottleneck. In many cases, in AI, your speed at which you can actually process information and create new large language models is really gated by the speed and availability of memory." - Steven Woo"More than 50 percent of the power is spent in circuits just trying to move data on and off the processor. It's pretty astounding to think that as companies plan how much power they need, a lot of it is really related to simply moving data back and forth." - Steven Woo"People think of compute in terms of gigawatts. But it turns out it's really the movement of that data - and nobody talks about that. It's the silhouette behind the curtain that's actually constraining everything else." - Seth EarleyTune in to discover why the future of AI depends as much on memory engineering as it does on model development - and what enterprise leaders need to understand about the infrastructure constraints shaping every AI investment they make.LinksLinkedIn: https://www.linkedin.com/in/stevencwoo/Website: https://www.rambus.comThanks to our sponsors:VKTREarley Information ScienceAI Powered Enterprise Book
