From severe storms to wildfires to climate patterns that unfold over decades, scientists understand the Earth system far better than they did in 2017. That’s due in no small part to the supercomputer named Cheyenne that began its well-deserved retirement last month.

In its seven years of operations, the flagship supercomputer at the U.S. National Science Foundation National Center for Atmospheric Research (NSF NCAR) was used by thousands of scientists across the country and overseas to advance knowledge into virtually every aspect of the Earth system.

"Cheyenne was an extraordinary success," said Thomas Hauser, director of the NSF NCAR Computational and Information Systems Lab (CISL). "Its leading-edge computing and data handling capabilities of its time enabled scientists to gain new insights into the Earth system. These insights continue to yield significant benefits for society."

NSF NCAR launched operations of the Cheyenne supercomputer in January 2017 at the NSF NCAR Wyoming Supercomputing Center (NWSC). As a 5.34-petaflops system, Cheyenne had the theoretical ability to perform 5.34 quadrillion calculations per second, which was triple the amount of scientific computing performed by its predecessor, the Yellowstone supercomputer. When inaugurated, Cheyenne ranked as the 20th fastest supercomputer in the world.

The system was originally slated to be replaced after five years. But then the COVID-19 pandemic struck, severely disrupting supply chains. Cheyenne was called upon to serve an additional two years until the necessary parts were available to build its successor.

Now the time has finally come for Cheyenne to be replaced by a more powerful system. The newest NSF NCAR machine, Derecho, is a 19.87-petaflops system that more than triples the performance of Cheyenne. It’s also more energy efficient than Cheyenne, which was three times more energy efficient than Yellowstone. Like other NSF NCAR computing systems, Derecho is available for use by the U.S. Earth system science community for research and instruction at no charge.

Even more powerful supercomputers are in NSF NCAR’s future. Hauser is already thinking about the systems to come, as he and his team consider the best architecture for increasingly data intensive work and how to further increase flexibility to support artificial intelligence applications. 

Cheyenne was built by Silicon Graphics International, or SGI (now part of Hewlett Packard Enterprise Co., which built Derecho). It was operated and maintained by a dedicated staff of about 20 engineers and other personnel at the NWSC.

By the numbers, Cheyenne was extraordinarily productive. It delivered more than 7 billion core hours, served more than 4,400 users, and supported nearly 1,300 NSF awards. It also played a key role in education, supporting more than 80 university courses and training events. Nearly 1,000 projects were awarded for graduate students and postdocs beginning their research careers.

Perhaps most tellingly, more than 4,500 peer-review publications, dissertations and theses, and other works resulted from NCAR high-performance computing system use since 2017.

A transformative system

Such numbers, however, tell only part of the story. What made Cheyenne especially valuable was how it shed light on the atmosphere and other facets of the Earth system in ways that will strengthen societal resilience.

High-performance supercomputers like Cheyenne enable researchers to run increasingly detailed models that simulate complex processes and how they might unfold in the future. Scientists can also harness the computing power to run multiple simulations, or ensembles, that enable them to quantify the probability of a given event. This is particularly helpful for providing resource managers and policy experts with needed intelligence for advance planning.

"There is no doubt that the Cheyenne supercomputer allowed me and my collaborators to more deeply address key scientific questions pertaining to wildfires, clouds, and turbulence,” said NSF NCAR scientist Tim Juliano, part of a team working to improve predictions of wildfires. “Without the generous resources provided by the NWSC, such insights would not have been possible."

In the case of Cheyenne, scientists said the supercomputing power proved transformative.

NSF NCAR scientist Falko Judt, who turned to Cheyenne for extremely high-resolution global simulations of the atmosphere, noted that the system played a vital role in advancing weather prediction techniques. For example, he used it to highlight an underappreciated type of rapid intensification of hurricanes, which will help forecasters better anticipate such events.

“The Cheyenne supercomputer has revolutionized our understanding of extreme weather,” he said. “It stands at the forefront of a weather forecasting revolution, providing unprecedented insights to foresee and respond to weather phenomena with precision and efficacy."

Cheyenne has also helped scientists learn more about the impacts of wildfire emissions on air quality and climate. A team of NSF NCAR scientists led by John Fasullo, for example, used Cheyenne to show that the smoke from massive Australian bushfires in 2019-20 had such significant atmospheric impacts that it helped generate a rare multi-year La Niña event, influencing global weather patterns. NSF NCAR scientist Rebecca Buchholz and her collaborators turned to Cheyenne to find that increasingly large and intense wildfires in the Pacific Northwest have altered the seasonal pattern of air pollution across North America, causing a late summertime spike in unhealthy pollutants.

 Scientists across the university community used Cheyenne to advance Earth system predictability. Ben Kirtman, atmospheric sciences professor at the Rosenstiel School of the University of Miami, worked with a team of researchers to develop a series of novel experiments with a high-resolution model that simulated ocean eddies. The goal was to improve longer-term prediction, which is highly influenced by ocean-atmosphere interactions.

“The Cheyenne supercomputer afforded us the unprecedented opportunity to test how resolved ocean-mesoscale processes affect terrestrial subseasonal-to-interannual forecast skill, and how these processes affect decadal variability and predictability,” Kirtman said.

Cheyenne was notable for enabling scientists to create large datasets and new modeling approaches. For example, NSF NCAR scientists led an effort to create a vast set of computer simulations to help identify the types of events that are most predictable years to as far as a decade in advance. The dataset, called the Decadal Prediction Large Ensemble (DPLE), contains a staggering 24,800 simulated years of climate information. Early analyses of the dataset found the potential to predict sea surface temperatures in the North Atlantic, which are tied to climate conditions across Europe and Asia, as well as potentially predicting multi-year precipitation anomalies over parts of Africa, Europe, and Eurasia.

Other examples of research with Cheyenne include:

• Future sea-level projections with a coupled atmosphere-ocean-ice sheet model. This advanced modeling study demonstrated that estimates of future sea level rise fundamentally depend on the complex interactions among ice sheets, icebergs, the ocean, and the atmosphere. A separate study looked at the likely extent of sea level rise by southeast Asian megacities.
• A hydrological simulation dataset of the Upper Colorado River Basin. This dataset provided a long-term and natural-flow simulation for one of the most over-allocated basins in the world.
• The development of a single, cohesive computer model to simulate the entire life cycle of a solar flare, from the buildup of energy thousands of kilometers below the solar surface to the emergence of tangled magnetic field lines and the explosive release of energy in a brilliant flash. The accomplishment set the stage for future solar models to realistically simulate the Sun's own weather as it unfolds in real time. 
• High-resolution modeling showing substantial increases in precipitation from storms associated with atmospheric rivers over California. The findings imply substantial challenges for water and flood management. A separate study warned that climate change is increasing the risk of a California megaflood.
• An hourly dataset for photovoltaic energy production for the continental United States. The dataset provided a high spatio-temporal analysis of the power production under different weather and engineering scenarios. A separate study looked at the impacts of wildfire smoke on solar energy production.
• A new method to apply statistical techniques to climate models in order to better estimate precipitation and runoff in mountain regions. This method can make it easier for water managers to use output from climate models to plan and adapt to climate change. 

“The remarkable breadth of research conducted on Cheyenne demonstrates the essential role of advanced supercomputing in understanding the Earth system,” said NSF NCAR Director Everette Joseph. “Such research helps answer fundamental questions about the world around us and provides decision-makers with the information they need to protect communities from events that could pose a threat to their lives and livelihoods.

“I am deeply grateful to the many staffers that kept Cheyenne in top operating order and enabled this important science,” Joseph added. “Cheyenne was a special machine that has certainly earned its retirement.”

See all News