The Datacenter Energy Crunch: More Opportunity Than Threat
Lately, I’ve spent some of my now-copious free time pondering the future of datacenters, suddenly pressured by AI to quicken their evolutionary pace. As you may have heard, AI is notorious for its voracious appetite for energy and its unquenchable thirst for water.
Today’s post is relatively brief, but I plan to address this topic more expansively in future correspondence. A broad swathe of innovations will contribute to datacenter resource efficiency, and their development will become increasingly important to an ecosystem that includes datacenter owners and operators, silicon designers and suppliers (CPUs, GPUs, NPUs, and network silicon), server and other system vendors, electrical utilities, and a constellation of renewable-energy and resource-optimization providers.
Contemporary market watchers are bedazzled by the exploits of the Magnificent Seven, but faster-growing investment vehicles, for venture capitalists as well as institutional and retail investors, are likely to be found in resources and technologies that deliver on the promise of energy-efficient and sustainable datacenters. Indeed, the ravenous energy consumption of datacenters hosting AI processing has sparked serious concerns about the availability and cost of electrical power.
A Reuters article published late yesterday, following proceedings at the AI: Powering the New Energy Erasummit, reported that U.S. electrical systems are not expanding fast enough to meet the rapidly growing needs of generative AI. The result is that datacenter owners and operators increasingly look beyond utilities to ensure an abundant and cost-effective energy supply.
What’s preventing electrical systems from meeting datacenter energy demands? According to the Reuters piece, cumbersome and labyrinthine regulations are part of the problem, compounded by often long-running legal imbroglios initiated by environmental and community groups. Consequently, much-needed energy projects have taken longer than expected to connect to the electrical grid, impairing the ability of electrical systems to meet rising energy demand from datacenters.
The following is an excerpt from the article:
"Regulation and permitting within the United States is abysmal," Brad Stansberry, who leads the financial management practice for the power and utility industry at services firm KPMG,
Datacenter developer Aligned, which is one of the biggest companies of its kind with 2.5 gigawatts of capacity, is hastily pursuing power supplies after spending several years focusing on buying land for its operations.
"We always assumed there would be power, but obviously that assumption was woefully inaccurate, so now we're chasing where power is," said Phill Lawson-Shanks, Chief Innovation Officer at Aligned.
Among its considerations for electricity, Aligned is looking at using small modular reactors, a type of nuclear energy system that is currently in development in the United States, to power some of its projects, and is working with utilities on projects when possible, Lawson-Shanks said. "Where we can't, we will have to have them behind the meter," he added.
SMR: An Acronym on the Rise
As mentioned in the Reuters piece and in articles cited in an earlier post, AI chatbots such as ChatGPT require ten times as much energy as a standard Google search. Before long, as more AI models are built and their subsequent inferences are accommodated, electrical systems, which lack the inherent agility and institutional flexibility of AI and cloud computing, will struggle to carry and deliver the load.
Small modular reactors (SMRs), mentioned above by Aligned executive Lawson-Shanks, are receiving considerable attention and even a measure of funding from significant players in the datacenter-infrastructure business. Microsoft is particularly active in exploring the nuclear option as means of powering its hyperscale datacenters.
Last September, DCD reported that Microsoft sought to hire a principal program manager of nuclear technology, who would "be responsible for maturing and implementing a global small modular reactor (SMR) and microreactor energy strategy." A year earlier, Microsoft, looking to address the energy needs of its datacenters, procured Clean Energy Credits (CECs) from Ontario Power Generation (OPG), a publicly owned utility that claims to be one of the largest clean-power producers in North America. OPG supplies more than 50% of Ontario’s electricity, primarily from low-emission sources such as hydroelectric and nuclear power. While the credits Microsoft purchased include power from conventional nuclear sources, they might later be applied to OPG’s future SMR deployment.
SMRs are neither exclusive to datacenters nor the only off-grid means by which hyperscalers and other datacenter operators will attempt to meet their burgeoning energy needs. Still, the SMR space is active and well funded. Companies developing SMRs include NuScale, TerraPower (whose founders include Bill Gates), Westinghouse Electric Company, BWXT (and its partner, GE Hitachi Nuclear Energy), Candu Energy Inc. (a subsidiary of AtkinsRealis, formerly SNC-Lavalin Group), as well as others. In China, a parallel SMR ecosystem is taking shape.
As mentioned, SMRs are just one of many technologies being developed to ensure that datacenters benefit from the availability, abundance, efficiency, and cost-effectiveness of sustainable energy. An assortment of intermittent renewable energy resources, as well as advanced energy-storage technologies, are also in development. This is to say nothing of technologies being developed to use water more efficiently, sparingly, and renewably.
Moreover, we can and should expect datacenter-infrastructure companies, especially silicon suppliers, to play their part in reducing the resource consumption of their technologies. Nvidia’s chips are not only expensive, but, despite generational advances, but also profligate in their resource consumption. Hyperscalers already had reason to reduce their reliance on Nvidia – a sole-source supplier is never considered advisable in a business where margins are extracted from highly efficient and prodigious scalability –so the opportunity to develop and use their own AI chips, with a more attractive energy (and operating cost) profile, enhances and strengthens the appeal of self-sufficiency.
Additional puzzle pieces, beyond silicon, will help to complete the complementary technological mosaic of future datacenters. Primary processing locations are likely to be chosen based on ready access to abundant renewable energy and water resources, architectural models overhauled to enact a novel interpretation of the well-established hub-and-spoke principles of content-distribution networks (CDNs) now used for streaming content, and all sorts of intra-datacenter technology breakthroughs are possible.
It's a big new challenge, and a larger opportunity, for a far-flung and diverse ecosystem.