HPE to Build 100+ Petaflops Shaheen III Supercomputer

The King Abdullah University of Science and Technology (KAUST) in Saudi Arabia announced that HPE has won the tender to build the Shaheen III supercomputer. Shaheen III, due to be fully operational next year, is expected to deliver 100 petaflops of Linpack from its accelerated partition alone, making it easily the most powerful supercomputer in the Middle East.

Shaheen III will consist of a pure computing partition and an accelerated partition. The compute-only partition, spread across 18 liquid-cooled HPE Cray EX4000 cabinets and networked with HPE Slingshot, will include 4,608 nodes, each containing two of AMD’s fourth-generation Epyc “Genoa” CPUs. KAUST does not yet publish performance estimates for the CPU cases – perhaps because providing estimates would allow a conclusion to be drawn about the performance of the Genoa CPU, which is not yet detailed. However, as a point of reference, if Genoa were to match the 3.58 FP64 teraflops of AMD’s current-gen Milanese CPU, the 9,216 AMD CPUs in Shaheen III would deliver about 33 peak petaflops.

Shaheen III will also feature seven accelerated enclosures (25 EX4000 enclosures in total). In these seven accelerated cabinets: 704 nodes, each with quadruple Nvidia “Grace Hopper” superchips, each with a tightly coupled CPU and GPU. Those 2,816 superchips will account for the bulk of Shaheen III’s flops: HPE says this partition alone will tip the scales at 100 petaflops from Linpack. Although Nvidia has yet to release teraflops for the superchips, based on this information we can reasonably speculate that each Grace Hopper superchip could deliver anywhere in the 35 Linpack teraflops range.

Based on accelerated partitioning alone, Shaheen III will be around 20x faster than Shaheen II, which debuted in 2015 at 5.54 linpack/7.24 peak petaflops (and was itself 25x faster than its predecessor). Accompanying this impressive system is a storage upgrade from Shaheen II: 50PB of Cray ClusterStor E1000 storage from HPE.

Shaheen II. Image courtesy of KAUST.

“A supercomputer like Shaheen III is a universal scientific instrument used by scientists and engineers in all disciplines for tasks such as simulation, analyzing experimental data, learning from observed data, and efficiently storing and retrieving data,” said David Keyes, director of KAUST Extreme Research Computing Center (ECRC) and Professor of Applied Mathematics and Computational Science. “It is the ultimate scientific ‘watering hole’ where researchers from different disciplines exchange techniques and software tools. Progress in one area spurs progress in several.”

(KAUST and Keyes landed recently HPCwire Reporting for her involvement in Reshaping Geostatistical Modeling and Prediction for Extreme-Scale Environmental Applications, one of six projects nominated for this year’s Gordon Bell Prize. The project used both Shaheen II and Fugaku.)

KAUST will use Shaheen III to address topics such as clean burning, Red Sea ecosystems, climate modeling, solar energy, healthcare, drug discovery, plant resilience, and the Arabian tectonic plate. More broadly, the supercomputer will enable KAUST to expand its simulation, analysis, and – most notably – its AI work, which will benefit from HPE’s machine learning development environment. The partners framed the investment in the new supercomputer in line with Vision 2030, a strategic plan to diversify the country’s economy.

“The new HPE Cray EX System will enable us to conduct research at a greater scale, leading to significant scientific, economic and social advances,” said Tony Chan, President of KAUST. “In line with Vision 2030, we strive to meet the ever-increasing demands of our active and solution-oriented faculty, as well as external partners, for faster and more efficient computing resources. KAUST’s supercomputing resources are used by more than half of our faculty, students, post-docs and researchers, as well as researchers from more than 20 external organizations in the Kingdom.”

With no emerging challengers, Shaheen III will become the most powerful (disclosed) supercomputer in the Middle East, superseding Saudi Aramco’s 22.4 petaflop Linpack Dammam 7 supercomputer, also housed in Saudi Arabia and also built by HPE.