As part of a daily routine, Cascade Technologies runs Large Eddy Simulation on many different computational facilities. Among them, the Intrepid system at Argonne is one of the two leadership computing facilities supported by the U.S. Department of Energy (DOE), and it currently stands at number 23 in the latest Top500 world ranking (November 2011).
Intrepid consists of 40 Blue Gene/P racks of 4096 cores each for a total of 163840 cores. A visual snapshot of the daily activity on Intrepid can be seen below, where each of the smallest squares is made up of 128 cores, every larger block represents 2048 cores (1/2 of a complete rack), and every color represents a different project:
Quite amazing isn’t it? Lots of activity and simulations running concurrently.
However, one of our jet noise, aeroacoustics Large Eddy Simulation runs on a 500 million unstructured grid; needing as much computing power as possible, we ended up using every single one of the 163840 available cores for 4 days in a row, for a total of 16 million core-hours of computational time.
Here is how the Intrepid activity screen looked during the simulation:
Definitely easier on the eyes in our opinion….
(our thanks to Joseph Nichols and Prof. Sanjiva Lele)