“Our primary goal is building an infrastructure for capturing, integrating, maintaining, and delivering data and knowledge to climate scientists.”
– Ghaleb Abdulla
Born at LLNL to address “big data” challenges in Earth system research, the Earth System Grid Federation (ESGF) is an international collaboration of computer scientists, data scientists, and climate researchers. Principal funding comes from the Department of Energy’s Office of Biological and Environmental Research with additional support from NASA and several foreign research centers including the European Network for Earth System Modelling, the German Climate Computing Centre, Australia’s National Computational Infrastructure, France’s Institut Pierre-Simon Laplace, and the U.K. Centre for Environmental Data Analysis.
The federation houses an enormous database of global observational and simulation data—more than 5 petabytes—and manages the high-performance computing hardware and software infrastructure necessary for scientific climate research. In the nearly two decades since its launch, ESGF has grown to serve 25,000 users on 6 continents. Co–principal investigator Ghaleb Abdulla notes, “ESGF’s primary goal is building an infrastructure for capturing, integrating, maintaining, and delivering data and knowledge to climate scientists.”
One of ESGF’s most significant accomplishments is supporting the Coupled Model Intercomparison Project Phase 6 (CMIP6) dataset. CMIP6 presented a significant test of ESGF’s infrastructure for the expected 20 petabytes of model output. Multiple components across the service stack needed to function under this level of stress—services such as publishing, search, download, and replication (i.e., moving data from one ESGF center to another).
As part of CMIP6 readiness, the input4MIPs initiative provided a key improvement in “forcing” dataset consistency when comparing with previous CMIP phases. Helmed by LLNL scientists, input4MIPs collects, archives, and documents climate datasets to support the coordinated modeling activities. ESGF hosts input4MIPs data alongside CMIP datasets, enabling climate researchers to evaluate climate models with uniform standards under the same conditions. LLNL’s Paul Durack won the 2018 World Climate Research Programme Data Prize for his leadership of input4MIPs.
Similarly, the obs4MIPs initiative began planning for CMIP6 in 2016. Co-led by LLNL and NASA and hosted on ESGF servers, this project established a database used by the CMIP modeling community for comparing satellite observations with climate model predictions. In 2018, the obs4MIPs team implemented several enhancements in data indicators and integration along with a prototype of color-coded quality indicators.
Team Acknowledgments
Pictured (left to right): Holly Davis, Chengzhu Zhang, Christopher Mauzey, Jason Boutte, Sasha Ames, Lina Muryanto, Ghaleb Abdulla, Jeff Painter, Nathan Carlson, Renata McCoy, Sterling Baldwin, Denis Nadeau, William Hill, Charles Doutriaux, and Carlos Downie.
Not pictured: Dave Bader, Anthony Hoang, Angela Jefferson, Sookyung Kim, Tanya Reshel, and co–principal investigator Dean N. Williams.
