3 December 2009  •  In a bright, quiet, air-conditioned room in eastern Tennessee, a portrait of Earth's progression from ice age to present is taking shape in unprecedented detail.  Each day, up to 100 years of climatic history unfolds.  By early this year, the story will be complete, thanks to some five million processing hours on two successive supercomputers at the U.S. Department of Energy (DOE) Oak Ridge National Laboratory.

NCAR's Bette Otto-Bliesner.

It's the first time that a climate model has portrayed such a long time span in such detail with oceans and atmosphere continuously interacting.  Even before the run is complete, papers are already emerging from the project, which is called Transient Simulation of Climate Evolution over the Last 21,000 Years (TraCE-21000).  It's supported mainly by the Paleoclimate Program of NSF and NCAR and employs the Community Climate System Model (CCSM), which is funded primarily by NCAR and DOE.

"To learn more about future warming, we need to unravel what happened in Earth's past," says NCAR's Bette Otto-Bliesner, who co-directs the project with Zhengyu Liu, of the University of Wisconsin-Madison (UW).  "This project is an important step toward better understanding of how the world's climate could change abruptly over the coming centuries."

The whole story comes into view

For all their power, supercomputers have limits, and the history of our atmosphere often tests them.  Paleoclimate modeling has long faced the constraints of computing power adequate for simulating aspects of the next century's climate but insufficient to depict much longer time periods from the past in detail.  One common solution has been to generate century-long "snapshots" that focus on key periods in past climate.  Similar to the preview to a Hollywood film, the approach gives scientists a few important scenes but omits the context surrounding them.

The other approach has been to carry out transient (continuous) runs at limited resolution, using so-called intermediate complexity models.  This provides some big-picture insight, but it may leave out some important processes, like the synopsis to a film that covers only the bare bones of the plot.

The TraCE-21000 project is examining the Bølling-Allerød, Earth's most recent period of natural global warming on a major scale, which took place about 14,500 years ago. Several factors combined to raise temperatures dramatically over Greenland and parts of Eurasia (red and orange). (Image by Jamison Daniel, National Center for Computational Sciences, Oak Ridge National Laboratory.)

Thanks to the special DOE computing allocation for TraCE-21000, scientists are now finally getting the equivalent of a full motion picture.  Among many advantages, the transient run enables direct comparison with long-term proxy data-extended reconstructions of temperature and other variables that are inferred from sediments, ice cores, and other sources.  Snapshot-style model runs are a challenge to sync with longer-term proxy records, but the continuous nature of the TraCE-21000 runs makes the task much easier.

TraCE-21000 collaborator Peter Clark (Oregon State University) mapped and dated deposits from the summit of Hawaii's Mauna Kea during the last glaciation.  The geologist found that Mauna Kea's ice cap showed a major advance at the same time-and likely in response to-one of the abrupt cooling events simulated in TraCE-21000.  (Photo by Mark Kurz.)

"The unique aspect of this project is being able to evaluate and better understand our conceptual thinking about how the system works through time, rather than at specific time periods," says Peter Clark, a geologist at Oregon State University.  "For the first time, we can compare the geologic data with model output to understand the physical mechanisms and linkages responsible for the global changes seen in the data."

To launch the experiment, scientists jump-started version 3.0 of the CCSM with known changes in Earth's orbit and in carbon dioxide concentration deduced from ice cores and other evidence. They then observed how the atmosphere and ocean responded. The results were in close agreement with temperatures, precipitation, and ocean circulation as deduced from fossil and geologic records.

TraCE-21000 has enlisted a wide range of specialists to ensure the fidelity of its depictions, including ocean, climate, geology, chemistry, and computing experts from a wide range of universities, federal laboratories, and nonprofit institutions. "It is a huge-and very successful-collaboration," says Liu.

Delving into the last big warming

The first major paper from TraCE-21000, published in Science on 17 July, explored the cause of a relatively sharp warming as Earth emerged in fits and starts from the extreme cold of the last ice age. Temperatures first rose during the interval from about 21,000 to 19,000 years ago, then cooled again in the Northern Hemisphere after glacial meltwater weakened the Atlantic Ocean's overturning circulation (sometimes called the conveyor belt). But in a subsequent period called the Bølling-Allerød warming, global sea level rose by some 5 meters (16 feet), and average temperatures in Greenland soared by up to 15°C (27°F) over a period of several hundred years.

Previous studies using simpler models had speculated that a sudden shift in the Atlantic conveyor belt might have caused the Bølling-Allerød warming. However, the new work with the CCSM suggests that a strengthening of the conveyor belt accounted for only about a third of the warming.  The rest is explained, according to the authors, by an increase of about 40 parts per million in atmospheric carbon dioxide and the release of heat stored in the ocean over thousands of years.

"Once the glacial melt stopped, the enormous subsurface heat that had accumulated for 3,000 years erupted like a volcano and popped out over decades," says Liu, lead author of the Science paper. "This huge heat flux melted the sea ice and warmed up Greenland."

A workshop in August at NCAR pulled TraCE-21000 participants together to discuss terrestrial vegetation and ecosystem dynamics, and more papers from the project are in the works.  One study is looking at the evolution of the hydrological cycle in the tropics, focusing on tropical Africa, where strong monsoons fed lush grasslands across the present-day Sahara thousands of years ago.  Another study is comparing the sensitivity of the Earth system derived from proxy data to that in the TraCE-21000 simulation.  Deep-ocean processes, including the release of ocean-stored carbon dioxide during deglaciation, are also being examined.

Meanwhile, at Oak Ridge, each week sees hundreds more years of climate coming to life.  According to DOE's David Erickson, a coauthor of the July paper in Science, "This collaboration between universities, NCAR, and DOE labs is a perfect example of how science can be advanced through high-performance computing."

A graduate student and his dream run

Not every doctoral student would be entrusted with keeping an eye on the most complex paleoclimate simulation ever conducted.  "It's scary, but also very exciting," says Feng He (left), who is carrying out the TraCE-21000 runs under the supervision of his advisor at the University of Wisconsin-Madison, Zhengyu Liu (right). "This is one of the dream runs for climate modelers."

Feng He and Zhengyu Liu. (Photo by Kathleen Barney.)

Among the ongoing challenges He faces is the need to adjust topography every 500 to 1,000 years of model time to account for ice sheet melt.  "That means I need to build a new model case and change both atmosphere and land model code," he says.  "There are also hundreds of runs I need to perform to test the sensitivity of the model to freshwater input, as well as the land-sea configuration due to sea-level rise from the retreating glaciers."  He has built more than 200 cases and processed close to 100 terabytes of data in his three years of TraCE-21000 work.

Like any grad student, He started out fearing the worst.  "The model could basically blow up or crash somewhere in the simulation, or I might mess up some code in the middle and not recall what I was doing wrong."  As it turns out, the process has gone relatively smoothly, for which He gives much credit to colleagues.  Spending a month at NCAR as the project ginned up, He worked closely with the center's Esther Brady and Robert Tomas.  He has also consulted with Tomas, NCAR's Samuel Levis, and DOE's Robert Jacob on issues involving dynamic vegetation and topography, and with NCAR's Lawrence Buja, Jon Wolff, Marianna Vertenstein, and Nan Rosenbloom on how to optimize the computing at Oak Ridge.

Putting a capable doctoral student like He on the project made more sense than using a postdoctoral researcher, according to Liu, since when all is said and done, the computational tasks will end up spanning more than the one or two years of the typical postdoc appointment.

"Feng is a very ambitious graduate student," says Liu.  "He complained at first that he was not getting a challenging enough problem for his Ph.D. thesis, so I figured this would be challenging enough."

See all News