NCAR selects IBM for key components of new supercomputing center
Nov 7, 2011 - by Staff
News Release
BOULDER—The National Center for Atmospheric Research (NCAR) today announced that IBM will install critical components of a petascale supercomputing system at the new NCAR-Wyoming Supercomputing Center (NWSC). The company was selected following a competitive open procurement process.
The building housing the NCAR-Wyoming Supercomputing Center in Cheyenne, Wyoming, was completed in the summer of 2011. (©UCAR. Photo by Carlye Calvin. This image is freely available for media & nonprofit use.*)
The IBM components consist of a massive central resource for file and data storage, a high performance computational cluster, and a resource for visualizing the data.
The new system, named Yellowstone, runs on an IBM iDataPlex and is expected to be delivered to the NWSC early next year. It will be the new facility’s inaugural system. Once installed, the system will go through a testing period before being made fully available for scientific research in the summer of 2012.
Yellowstone is expected to deliver 1.6 petaflops performance, or nearly 30 times the capacity of the system currently in use at NCAR’s Mesa Laboratory in Boulder, known as bluefire. Petaflops refers to a machine’s ability to perform one quadrillion calculations, called floating point operations (FLOPS), per second.
Scientists will use these advanced computing resources to understand complex processes in the atmosphere and throughout the Earth system, and to accelerate research into climate change, severe weather, geomagnetic storms, carbon sequestration, aviation safety, wildfires, and other critical geoscience topics.
“Yellowstone will provide needed computing resources to greatly improve our understanding of Earth and produce significant benefits to society,” says Anke Kamrath, director of operations and services for NCAR’s Computational and Information Systems Laboratory (CISL). “We are very pleased to have such a high-performance system inaugurate the new supercomputing center.”
“The vision for Yellowstone parallels the principles that have guided the design of the NWSC,” says NCAR director Roger Wakimoto. “In both instances, we have taken an approach that maximizes the science we can do and the benefit of that science to society.”
The NWSC is the result of a partnership between NCAR; the University of Wyoming; the State of Wyoming; Cheyenne LEADS; the Wyoming Business Council; Cheyenne Light, Fuel and Power; and the University Corporation for Atmospheric Research. NCAR is sponsored by the National Science Foundation.
Inside the NCAR-Wyoming Supercomputing Center, a 12,000-square-foot computer room stands ready to receive its first components, which will occupy about one-third of the space. The small, temporary "loadbanks" shown here have been used to test the room's readiness to handle electrical and heat loads and will be removed when the new IBM system arrives. (©UCAR. Photo by Carlye Calvin. This image is freely available for media use. For more information, see Media & nonprofit use.*)
The facility, located in the North Range Business Park in Cheyenne, Wyoming, was designed with sustainability and flexibility in mind so it can be easily adapted to future technologies and changing requirements in scientific computing. Construction and commissioning was completed last month.
"We join our partners at NCAR and in Cheyenne in great anticipation as the supercomputer project nears completion," says University of Wyoming president Tom Buchanan. "Our faculty are poised to take full advantage of all that the center will bring to our educational and research endeavors in Earth system sciences; atmospheric, hydrological and computational sciences; mathematics; and engineering. The outcomes will enhance science and technology throughout Wyoming and the nation.”
Details of the new system
Yellowstone is an IBM iDataPlex supercomputer system, consisting of Intel Sandy Bridge EP processors and a Mellanox FDR InfiniBand full fat tree. It will have 149.2 terabytes of memory, 74,592 processor cores, and a peak computational rate of 1.6 petaflops.
The central file and data storage resource will consist of file system servers and storage devices that are linked to the data analysis and visualization (DAV) resources and to the supercomputer systems. These centralized file systems will allow scientists to generate model output on the supercomputer and then analyze or visualize it on the DAV resource, without the usual bottleneck when moving such large quantities of data between separate systems.
Yellowstone’s central file system will have nearly 17 petabytes of usable disk space, 12 times what is available to NCAR’s scientific research community today.
The DAV resource is made up of two systems, one designed to facilitate large-scale data analysis, and the other for parallel processing and visualization activities.
Taken together, these components will dramatically improve capabilities central to NCAR’s mission, such as climate modeling, forecasting, and preservation of critical research data. The NWSC will serve researchers across the United States and around the world who will interact with its systems remotely.
“NCAR has a longstanding tradition for excellence in deploying resources to address unique challenges requiring high-performance computing capabilities,” says David Jursik, vice president of high-performance computing at IBM. “Yellowstone will substantially expand NCAR's ability to investigate climate change, severe weather events, and other important subjects. We are pleased to be selected as IBM continues to pursue high-performance computing for the purpose of helping scientists and leading decision makers address critical issues for a smarter planet.”
CISL director Al Kellie emphasizes the importance of the integrated computing resource, explaining that what makes this system exceptional for geoscience research is the linking of a very large centralized file and data storage system to a high-performance computational cluster and visualization resource.
“While we wanted to make sure we had adequate computing capacity, we knew that it would be of limited use if we didn’t ensure easy access to the data and appropriate resources for storing and analyzing it,” Kellie says. “In addition to high-performance machines, researchers need quick access to their data and a way to analyze it, to see what it means. This system addresses those needs elegantly.”
Quick facts
- One petaflop is one quadrillion (1,000,000,000,000,000) floating point operations per second, which is approximately 143,000 calculations per second for every man, woman, and child on Earth.
- At 1.6 petaflops, Yellowstone will be capable of more than 221,000 calculations per second for every person on Earth.
- The Yellowstone system features 9.7 million times the computational rate, 3.4 million times the disk capacity, and 19 million times the central memory size of one of the world’s first supercomputers, the Cray 1-A, which supported NCAR’s computational science between 1977 and 1989.