A massive dataset that is widely used by the climate and atmospheric research community to study phenomena like extreme rainfall, ice loss, and extended droughts in North America, is now freely available on the commercial cloud. 

The National Center for Atmospheric Research (NCAR) has made a significant portion of climate model simulations available to the scientific community via Amazon Web Services (AWS). The data come from the NCAR Community Earth System Model (CESM) Large Ensemble Numerical Simulation (LENS), which is intended to advance understanding of climate change and variability.

“The scientific community has widely used the CESM LENS data already, but making them accessible from the cloud enables new users to explore research possibilities and computing frontiers,” said Jennifer Kay, a professor at the University of Colorado Boulder and co-leader of the CESM LENS project with NCAR scientist Clara Deser.

CESM LENS is a 40-member ensemble of climate simulations from the years 1920 to 2100. The total data volume is large, more than 500 terabytes, which has been and continues to be accessible to download through the NCAR Climate Data Gateway, or for analysis by users who have accounts on the NCAR-Wyoming Supercomputing facilities. Of the overall ensemble, NCAR has made approximately 70 terabytes of compressed data freely available on AWS, including the most widely used variables like temperature, precipitation, land surface, and ocean surface. 

“This dataset is too large to easily copy. This effort enables both the academic community and private industry who lack access to the NCAR supercomputer to perform data analysis directly in the cloud,” said Jeff de La Beaujardière a division director at NCAR and project manager of this new data accessibility initiative.

The data have been optimized for faster analysis than can be done within the Amazon cloud, so users are not limited by the storage or computing capacity of their own institutions. 

The CESM LENS dataset is currently being used in hundreds of research projects and has been cited in 580 scientific journal articles. Projects have used CESM LENS data to study increased drought risk in California, climate change-driven sea level rise, and the influence of global warming on extreme climate events. With new opportunities to perform data analysis in the cloud, the CESM LENS project leaders hope to see new users of the large ensemble.

“The CESM1 large ensemble has been such a success,” said Kay, who is also a visiting scientist at NCAR. “I am stunned by the diversity of applications and broad international use. This project continues to spark new discoveries and new interactions between science and industry.” 

Additional information:

Information about accessing CESM LENS on AWS, code repository, available data, and more can be found on the dataset landing page. For a description of CESM LENS and initial results, visit the published paper in the Bulletin of the American Meteorological Society. Additional information can be found on the project website.

See all News