A new milestone in high performance computing was reached late Tuesday evening (1/22/13) when Stanford researcher and Cascade consultant Dr. Joe Nichols ran the CharLES solver on more than 1 million processor cores. This breakthrough happened during “Early Science” testing of the newly installed Sequoia supercomputer at the Lawrence Livermore National Laboratories (LLNL). The Sequoia IBM Bluegene/Q system is currently ranked No. 2 on the list of the world’s most powerful supercomputers boasting 1,572,864 compute cores and 1.6 petabytes of memory connected together with a high-speed five-dimensional torus interconnect. A CFD simulation tasks all parts of a supercomputer because waves propagating throughout the tightly-coupled simulation require a well-orchestrated balance between computation, memory, communication, and I/O.
A snapshot of pressure and temperature from the “crackling” jet run during the scaling study.
At the one million core level, previously innocuous parts of simulation codes may suddenly become bottlenecks, and massive parallelism through all aspects of the software architecture is critical. Joe and other researchers from Stanford’s PSAAP program and LLNL computing staff have been working closely together for a few weeks now to prepare for this unprecedented opportunity. So, together, they were glued to their terminals (more than usual) Tuesday afternoon and into the evening during the first “full-system scaling” window of the early science testing period to see whether initial extreme-scale science runs would achieve stable run-time performance after starting up. As the first CFD simulation passed successfully through its initialization phase, all were thrilled as they saw the code performance continue to scale all the way to and beyond one million cores. This means that the time-to-solution continued to reduce, enabling more science to be done with ever faster turnaround times. These early science runs represent at least an order-of-magnitude increase in computational power over the largest runs performed using CharLES to date, enabling unprecedented fidelity and dramatically reducing time to solution.
Calculations were performed on the Sequoia supercomputer installed at Lawrence Livermore National Lab.
[...] article from Cascade Technology [...]