NCAR to enhance data analysis tools with input from broad scientific community
Organization shifts to open development to harness the community’s creativity
Oct 30, 2020 - by Laura Snider
A visualization of smoke from the wildfires burning on the U.S. West Coast on Aug. 30, 2020. The image, which looks east, was generated by VAPOR using data from NOAA's HRRR-Smoke model. The visualization was produced by Ka Yee Wong at NOAA.
The National Center for Atmospheric Research (NCAR) is inviting scientists worldwide to help further develop two of its premiere geoscience data visualization and analysis software programs: GeoCAT (Geoscience Community Analysis Toolkit) and VAPOR (Visualization and Analysis Platform for Ocean, Atmosphere, and Solar Researchers).
The tools will enable scientists to gain insights into increasingly complex datasets and better understand crucial Earth system processes. The decision to shift to open development follows in a decades-long NCAR tradition of using open-source codes and working with the research community to improve world-class models of weather, climate, water, and other aspects of the Earth system.
GeoCAT, launched earlier this year, is the successor to NCAR Command Language (NCL). Since it was first released in 1995, NCL has become a foundational tool for researchers in Earth system science. Its users, which have grown to 100,000, rely on NCL for post-processing and to create high-quality, customized visualizations to help interpret their data.
NCAR decided last year to discontinue further development of NCL in favor of transitioning to tools programmed in Python, a language that is increasingly used for a wide variety of scientific applications and which more and more early career scientists are already fluent in. It can be used more broadly than NCL, which is highly specialized for geoscience applications.
“Python can replicate much of the functionality of NCL and provide other critical features not available in NCL,” said NCAR software engineer Orhan Eroglu, who is now leading the development of GeoCAT. “It offers significant advantages and has already been rapidly adopted by the Earth system science community.”
Eroglu’s group has been hard at work transferring the specialized functionalities in NCL, of which there are hundreds, to GeoCAT. They have also been building out a gallery of GeoCAT plotting examples that show users exactly how the NCL functionality can be replicated in Python, including boxplots, map projections, regressions, and much more.
The scale of the project is also why NCAR is inviting help from the community. NCAR developers want to know what functionalities from NCL that have not been replicated yet are important to the community, and they also invite people to contribute to the open source code directly through the software collaboration platform GitHub.
“The tools developed by NCAR, from models to data sets to software packages, have always been built in collaboration with the community and made available for anyone to use,” said John Clyne, the manager of NCAR's Visualization and Analysis Software Tools (VAST) section. “The trend in software development to move from open source to open development – and the availability of platforms like GitHub to facilitate this shift – is making it easier for us to involve the community directly in the development of our heavily used analysis toolkits.”
NCAR also released an updated version this year of VAPOR, a visualization and analysis toolkit that doesn’t require the user to know a programming language. VAPOR is a point-and-click application and is better suited to 3D visualization than NCL or GeoCAT. It’s useful for exploring datasets when you’re not yet sure what you are hoping to find, Clyne said. VAPOR is also up on GitHub, enabling scientists to contribute to further code development.
“GeoCAT and VAPOR are both powerful visualization tools for researchers,” Clyne said. “And we’re excited that both continue to grow and become better with the help of the community.”