CHEYENNE, Wyoming — The National Center for Atmospheric Research (NCAR) has launched operations of its newest supercomputer, providing scientists across the country with a major new tool to advance understanding of the atmosphere and other Earth system processes.

The supercomputer, named “Derecho,” is a 19.87-petaflops system, meaning it has the theoretical ability to perform 19.87 quadrillion calculations per second. That is about 3.5 times the speed of scientific computing performed by the previous NCAR supercomputer, Cheyenne, and represents the equivalent of every person on the planet solving one calculation every second for a month. Derecho is also the first NCAR supercomputer to include a significant number of graphics processing units (GPUs), with 382 NVIDIA A100 GPUs providing 20% of its sustained computing capability.

Scientists across the country are starting to use Derecho to study phenomena ranging from wildfires and hurricanes to solar storms. Their findings will help better protect society from environmental disasters, lead to more reliable projections of long-term weather patterns, and improve weather and climate predictions that are needed by vulnerable communities and critical sectors of the economy such as agriculture and transportation.

Derecho was built by Hewlett Packard Enterprise (HPE). It was installed earlier this year at the NCAR-Wyoming Supercomputing Center (NWSC) in Cheyenne, where it has undergone months of testing.

The new supercomputer is so energy efficient that it will use only about 40% more electricity than Cheyenne – which is itself highly energy efficient – despite being 3.5 times faster. This is partly because GPUs can deliver more computing power for less energy than traditional central processing units (CPUs), and they are also particularly effective for newly developed artificial intelligence and machine learning techniques.

“We are very pleased that Derecho has passed through the testing phase and is beginning operations,” said Thomas Hauser, interim director of NCAR’s Computational and Information Systems Laboratory. “This new system provides us with a major boost in supercomputing performance and is an invaluable asset as the nation’s scientists work to better understand the Earth system.”

Funding for Derecho, which cost about $35 million, came from the U.S. National Science Foundation (NSF). The NWSC is funded by NSF and the state of Wyoming through an appropriation to the University of Wyoming.

A more complete picture of the Earth system

Derecho will initially be used for a set of 15 projects by scientists at NCAR as well as at universities across the country. These comprise the Accelerated Scientific Discovery (ASD) program, which provides a unique opportunity for large-scale computational projects to have nearly exclusive use of new NCAR high-performance computing systems for a few months. Derecho will be open to additional projects by the scientific community later this summer.

Some of the ASD projects include:

Climate change in the West and the Arctic. Scientists are applying a technique known as downscaling to create an ensemble of detailed computer simulations showing how climate is likely to change at very fine scales across the western United States. This novel approach, led by Alex Hall of the University of California, Los Angeles, will help reduce uncertainties about future climate and provide researchers with an important data set at a time when the West is contending with unprecedented water shortages, fire weather conditions, and heat extremes. Another team of scientists, led by Keith Musselman of the University of Colorado Boulder, is producing a set of climate simulations to strengthen understanding of changes in Arctic hydrology and potential impacts on rivers, fish, and Indigenous communities in Alaska and the Yukon. That research is being guided by an Advisory Council of Indigenous leaders and representatives,

Geoengineering. To potentially offset global warming, scientists are looking into methods such as injecting particles into the stratosphere to reduce incoming solar radiation. Such efforts to artificially cool the planet, however, may have unintended consequences on Earth’s climate. A team of scientists, led by Kristen Rasmussen of Colorado State University, will run advanced climate models on Derecho to assess the influence of climate change on showers and thunderstorms in South America and determine how stratospheric injection might influence those storms.

Hurricanes. As scientists seek to improve hurricane forecasts, a major question is the influence of sea spray droplets on these powerful storms. David Richter of the University of Notre Dame is leading a project that uses Derecho’s GPUs to better understand the movement of airborne droplets and the extent to which they can be carried aloft in extreme conditions. NCAR’s Rosimar Rios-Berrios and colleagues will use Derecho to explore a different question about hurricanes: will the warming climate lead to more instances of catastrophic hurricane rainfall? To answer that question, Rios-Berrios will focus on simulations showing interactions between clouds and solar heat and the extent to which those interactions affect hurricanes and precipitation extremes.

Wildfires and air quality. Wildfire smoke has a major impact on air quality, both locally and on areas far downwind. NCAR scientist Timothy Juliano is leading a project to simulate the massive 2020 California August Complex fires. By running a specialized, high-resolution computer model capable of capturing wildfire-chemistry-atmosphere interactions on Derecho, the research team will examine emissions and plume rise from the fire, gaining a complete picture of wildfire smoke generation, transport, and evolution. Another project, led by NCAR’s Louisa Emmons, will examine the impact of long-range transport of pollutants and regional influences on urban air quality worldwide and, in turn, the influence of local air quality on the global atmosphere.

Space Weather. Solar storms can have far-reaching impacts on Earth, affecting communications and navigation systems or disrupting the power grid. Major solar eruptions often originate from active regions that are composed of several bipolar sunspot groups that interact with each other. NCAR scientist Matthias Rempel will lead a project using Derecho’s GPUs to simulate a 2011 event in which opposite polarities collided and formed a powerful solar flare. The investigation will focus on the build-up and release of magnetic energy. Another space weather study, led by Mikhail Sitnov of Johns Hopkins University, will use Derecho to examine magnetotail reconnection on the night side of Earth’s magnetosphere, a process that injects energy towards Earth and can result in large electromagnetic disturbances. The research, using new empirical models of the magnetosphere based on spacecraft observations, seeks to address two questions: how exactly does the plasma and magnetic field transit from the reconnection site to the inner magnetosphere and how is the site of reconnection determined?

Long-range prediction. A pair of studies will use Derecho to enhance long-range predictions of weather and climate. One project, led by Maria Molina of NCAR and the University of Maryland, College Park, will use a data-driven, deep learning approach with Derecho’s GPUs to create a 100-member ensemble of subseasonal forecasts of global precipitation and temperature. The research can help lead to improved predictions of rainfall patterns and heat waves several weeks in advance. A second study, led by NCAR’s Ben Johnson, will analyze ocean models used to simulate regions that are important for the Pacific Decadal Oscillation and other ocean-atmosphere phenomena that influence global climate. Such research can enable scientists to better understand these phenomena, which are critical for long term prediction.

For more about these and other projects, see the NCAR Computational & Information Systems Lab webpage.

See all News