Nuclear fusion is a process that creates energy by smashing two atoms together. Scientists believe a fusion-based power plant could supply almost limitless energy while creating very little waste.
The problem: No one knows whether it’s safe to create the superhot gas needed to trigger fusion.
“We’re using 95 percent of the capacity of these computers, and we aren’t close to realism,” says Amitava Bhattacharjee, a professor of astrophysical sciences with the Princeton Program in Plasma Physics. ”
Simulating fusion, and accounting for millions of variables and particle interactions, requires an “exascale” system that can perform one quintillion calculations every second, roughly the combined processing power of the 500 fastest supercomputers now in existence.
This need for high-powered simulations is shared by many scientific disciplines, ranging from nanoscale particle physics up to cosmology.
“These problems all involve complex underlying equations,” Bhattacharjee says. “We need exascale simulations in order to be able to make predictions in these systems where there’s a lot going on.”
The U.S. Department of Energy expects an “exascale” computer like this could lead to dramatic advances in precision medicine, quantum physics, and biofuel development. In early June, the department announced it selected HPE to develop the architecture for exascale computing under the DOE’s Path Forward program. The program aims to deliver its first exascale supercomputer by 2021.
“Reaching exascale means achieving over 10x improvement in computing capacity, without increasing the power consumption, in less than five years”
Building an exascale computer isn’t simply a matter of adding more processing cores to a system, the classic path to more speed. At a certain point, computers stop being able to move data from memory to processors, and from processor to processor, fast enough to take advantage of the raw power.
“In the halls we say ‘processing is free,’” says Mike Vildibill, vice president of advanced technologies at HPE. At exascale speeds, the cost of moving data exceeds the cost of computation, he says. “And in exascale computing, data movement is king.”
Vildibill is part of a team of HPE engineers and researchers working on the exascale computer. He and his colleagues are designing a system with as many as 100,000 processors, capable of performing one quintillion calculations per second.
To get here, they’ve scrutinized every bottleneck to data movement in today’s computing technology, identifying better ways to transmit data over novel optical interconnects, shorten the distances data travels, and route data more efficiently inside the system. They’re also figuring out how to make many of the underlying components necessary to achieve exascale.
One of the biggest challenges: moving data around a supercomputer uses a lot of power and generates enormous heat. An exascale system built with today’s technology would consume about 650 megawatts of power—or a little less than a small nuclear power plant generates.
“You’d practically need to build a dedicated power plant right next door just to operate the system,” Vildibill says. Even if that was practical, the heat from all that energy would cripple the system. ”
To make data go faster while cutting power usage, the team has re-thought nearly every component and system inside today’s supercomputers. Here are four innovations that will enable exascale computing.
Inside an exascale computer
Memory and processors together
Each processor in a computer requires a dedicated chunk of memory, called its working memory, to perform calculations. The communication bandwidth between the processor and its working memory has a big impact on overall system speed. Today’s systems generally house memory in components called DIMMs, or double in-line memory modules, which connect to the processors via copper wiring in the circuit board. Exascale systems will need to move the vast majority of working memory far closer to the computing elements, in an organization called “co-packaged memory”, where the fastest memory tier is designed inside the physical package that also contains the processor chip.
“In the next generation of semiconductors we’ll see processors with computing power of well over 10 teraflops per second,” says Paolo Faraboschi, an HPE fellow and vice president. “To fully utilize those processors, you need at least one-tenth of that amount in bandwidth between the chip and memory, or one terabyte per second. This is very challenging with traditional discrete memory technology.”
“An exascale system built with today’s technology would consume about 650 megawatts of power—or a little less than a small nuclear power plant generates.”
The HPE approach is straightforward: Co-packaged components mean data travels only a short distance between processor and memory, rather than moving a longer distance between separated components. This silicon-connected memory replaces DIMMs and also eliminates the overhead of attaching routing information to the data to notify the system where that data is headed.
4 places data movement slows down today's computers
Silicon traffic cops
Data moves between processors and other components as they coordinate their work on big calculations. Inside a large-scale supercomputer this movement is directed by fabric routers, chips that act like traffic cops for data. Without efficient routing, data can be sent on a circuitous path or collide with other data trying to use the same path. Routers are also responsible for tolerating network failures by dynamically adjusting paths to go around faults.
“Basically, high-radix means fewer hops,” says Al Davis, a distinguished technologist at Labs. In turn, fewer hops means data can arrive at its destination faster. The HPE design calls for no more than four hops for data to reach any of the processors from any other point in the exascale system—even if the number of processors reaches 100,000. ”
The inner topology
Routers are part of a broader data transmission fabric inside the system. The pattern in which these connections are laid out is called a topology, and the topology determines how efficiently data inside a system reaches its destination.
Optical components and fabric
Copper has served faithfully as the conductor of data—transmitted as an electrical signal—since the 1940s. At exascale computing speeds, though, copper’s shortcomings move to the forefront. The more data, the more electricity required to move it.
“In earlier times, there were two pillars of scientific discovery: experiment and theory. Now computer simulation is becoming the third pillar”
In current optical architectures, each transmission cable needs its own VCSEL, and the system needs a huge number of cables to achieve a resilient optical fabric of interconnected components.
Wait, there’s more
Exascale computing, and data movement in particular, presents plenty of other challenges. For example, sometimes the most efficient solution is to not move data at all. Instead of copying a big block of data from one area of memory to be closer to a distant processor, an exascale system might sometimes just want to tell that processor where the data is located—giving it the right address—and let the processor order up a specific calculation based on that data without moving the data itself.
“Simulation is the bridge between what we know today and what we can predict in the future. And simulation is what exascale computing is all about.”