The JBGeorge Tech Blog

.

This is a duplicate of a high-performance computing blog I authored for Cray, originally published at the Cray Blog Site on Nov 19, 2019.

In April, we announced three offers to expand Cray supercomputing in the cloud capabilities for customers: Cray ClusterStor in Azure (for HPC storage), Cray in Azure for Manufacturing, and Cray in Azure for Electronic Design Automation.

We’ve seen strong interest in all three offers among a number of customers over the past months, all seeking out ways to run HPC and AI jobs on Cray in Azure. But we’ve seen particularly intense interest in the Cray ClusterStor in Azure offer.

Managing data storage and its accessibility is a common challenge among nearly all of these organizations, so it’s not a total surprise. HPC jobs at scale tend to generate a significant amount of data, and that data is critical for continued evaluation and processing.

ClusterStor in Azure is particularly attractive to customers looking for HPC storage options in the cloud because it helps meet those needs, and it’s an incredibly scalable, competitively priced option for high-performance storage. It offers a Lustre®-based, single-tenant, bare metal, and fully managed HPC environment in Microsoft Azure, and is available in easy-to-consume small, medium, and large configurations. ClusterStor in Azure delivers the exact performance, speed, scalability, data protection, and availability you need.

But, how would Cray ClusterStor in Azure perform?

ClusterStor typically performs great in data centers, but how would it respond in a cloud data center? The power envelope is different, the network technology is different — along with numerous other variables that change going from an on-premise to a cloud data center.

As a test, Cray and Microsoft worked with customers on a use case to test this performance. Our chosen use case included imaging, modeling, and simulation. We decided that the best way to test ClusterStor in Azure would be to independently measure read-performance and write-performance.

The Azure configuration was designed to simulate imaging jobs utilizing a variety of pre-stack and post-stack migration, full-waveform inversion, and real-time migration techniques.

Specifically, here are the specs for what we started with:

• 468 Azure HB VMs totaling over 28,000 AMD® EPYC® 1st gen CPU cores

• More than 123 TB/sec aggregate memory bandwidth

• Cray ClusterStor L300

We’ve seen ClusterStor excel over and over again in a variety of use cases, so we expected excellent results.

And we were right!

Here’s what we found:

• The combination of the Azure HB-series VMs and Cray ClusterStor storage provided a highly scalable system delivering an 11.5x improvement in time to solution as the pool of compute VMs was increased from 16 to 400.

• Cray ClusterStor performance peaked at 42 GB/sec (reads) and 62 GB/sec (writes). It also delivered significant differentiation by driving a 66% improvement in application performance as compared to an alternative, high-performance NFS approach.

• In fact, Cray ClusterStor in Azure achieved performance over an Ethernet network that rivals that of the dominant, proprietary HPC interconnects, saving cost without sacrificing performance!

Impressive results indeed — which is exactly what you should expect when you choose the powerful combination of supercomputing and cloud capabilities for the mission critical applications of your organization.

Learn more about Cray ClusterStor in Azure here or talk to your rep to work out a Cray in Azure configuration that best fits your cloud strategy.