Volume 39

On July 16, the Department of Energy (DOE) formally announced the proposed Frontiers in Artificial Intelligence for Science, Security and Technology (FASST) initiative via the web page www.energy.gov/fasst (with accompanying video and fact sheet). As stated on the web page, the speed and scale of the AI landscape are significant motivators for investing in strategic AI capabilities: “Without FASST, the United States stands to lose its competitive scientific edge and ability to maintain our national and economic security, will have a less diverse and competitive innovation AI ecosystem, will not have the independent technical expertise necessary to govern AI, and will lose the nation’s ability to attract and train a talented workforce. Through FASST, we will meet the mission needs of national security, energy security, and scientific discovery that will support sustained economic prosperity for the nation for decades to come.”

FASST has five goals: advance national security; attract and build a talented workforce; harness AI for scientific discovery; address energy challenges; develop technical expertise necessary for AI governance. Additionally, the initiative will build the world’s most powerful integrated scientific AI systems through four key interconnected pillars: AI-ready data; frontier-scale AI computing infrastructure and platforms; safe, secure, and trustworthy AI models and systems; AI applications.

Four Livermore co-authored papers were accepted to the 2024 International Conference on Machine Learning, one of the world’s prominent AI/ML conferences. Two of the papers examined trustworthiness—how a model uses data and makes decisions—of large language models, or LLMs. (A summary of the other papers will be in next month’s newsletter.)

In “TrustLLM: Trustworthiness in Large Language Models,” Bhavya Kailkhura and collaborators from universities and research organizations around the world developed a comprehensive trustworthiness evaluation framework. They examined 16 mainstream LLMs—ChatGPT, Vicuna, and Llama2 among them—across 8 dimensions of trustworthiness, using 30 public datasets as benchmarks on a range of simple to complex tasks. Led by Lehigh University, the study is a deep dive into what makes a model trustworthy. The authors gathered assessment metrics from the already extensive scientific literature on LLMs, reviewing more than 600 papers published during the past 5 years. Spoiler alert: None of the tested models was truly trustworthy according to TrustLLM benchmarks.

In “Decoding Compressed Trust: Scrutinizing the Trustworthiness of Efficient LLMs Under Compression,” James Diffenderfer, Brian Bartoldson, and Kailkhura joined colleagues from several universities to investigate trustworthiness in the context of compression, where a model is modified to reduce the amount of data and compute resources necessary for efficiency. The team applied five compression techniques to leading LLMs, testing the effects on various trustworthiness metrics, and discovered that compression via quantization was generally better—i.e., the model scored higher on trust metrics—than compression via pruning. Furthermore, they saw improved performance of 4-bit quantized models on certain trustworthiness tasks compared to models with 3- and 8-bit compression.

The rapid pace of LLM development raises new questions even as researchers answer existing ones. And with growing emphasis on this technology among the AI/ML community and at top conferences, understanding how LLMs work is the key to realizing their potential. Read more about both papers via LLNL Computing.

LLNL welcomed students from four institutions for this year’s Data Science Challenge (DSC) internship program. Hosted by the DSI, the DSC gives undergraduate and graduate students a taste of the multidisciplinary research performed at national laboratories. In addition to UC Merced and UC Riverside, participants hailed from two new partnering institutions: Case Western Reserve University (CWRU) and California State University, Bakersfield (CSUB). Held at the University of California Livermore Collaboration Center, the intensive two-week internship offers students hands-on experience using data science techniques to address real-world computational challenges. Livermore scientists, engineers, and technical staff instruct and mentor students as they navigate challenge problems involving data gathering and analysis.

Expanding the program to include additional schools is a key part of the DSI’s outreach strategy, and this inaugural experience with CWRU and CSUB has been a success. Jane Dong, dean of CSUB’s School of Natural Sciences, Mathematics, and Engineering, says, “Many of our students are first-generation college attendees from low-income families. The prospect of learning from world-class scientists and applying data science principles to drive innovation is incredibly enticing for them.” 

On August 4, LLNL researchers Felipe Leno da Silva and Ruben Glatt hosted the AI for Critical Infrastructure workshop at the 33rd International Joint Conference on Artificial Intelligence (IJCAI) in Jeju, South Korea. Professors Wencong Su (University of Michigan – Dearborn) and Yi Wang (University of Hong Kong) joined them in organizing the workshop focused on exploring AI opportunities and challenges in this globally important domain.

“Given the fast pace of advances in AI, collaboration with industry and academia is required to remain in the forefront of AI developments. Hosting this workshop at a top AI conference not only supports keeping LLNL updated about external advances but also promotes forging new partnerships,” says Leno da Silva, who serves as DSI’s technical outreach coordinator.

Critical infrastructures (CIs) encompass essential public services such as transportation, energy, telecommunications, hospitals, and information technology networks. AI can optimize CI planning processes, forecast future demands, and pinpoint vulnerabilities. During operations, AI can ensure seamless functioning by quickly addressing disruptions and mitigating risks such as cyberattacks or natural disturbances intensified by climate change. Despite the potential benefits, the application and evaluation of AI in CIs are complex due to their critical impact on society. Ensuring the safety and trustworthiness of increasingly autonomous AI systems in CIs is crucial, necessitating robust frameworks to safeguard public trust and infrastructure integrity.

To further engage with the research community, the organizers are working with Springer Nature to publish a special issue journal on AI for CI topics. Workshop papers will have a shortcut opportunity to submit an extended version of their paper, and new submissions are accepted as well until November 15. For more information about the publication opportunity, contact leno [at] llnl.gov (leno[at]llnl[dot]gov).

In the enduring quest to speed up scientific simulations without sacrificing accuracy, physics-informed neural networks (PINNs) have emerged as a meshless alternative to numerical methods for approximating partial differential equation (PDE) solutions. PINNs learn physical equations in addition to the training dataset, which means this type of ML could produce meaningful results with suboptimal data. Despite this promise, PINNs struggle on time-dependent problems such as fluid advection and wave dynamics.

The issue is error propagation. PINN systems typically converge at each time step before moving to the next—either by retraining one NN sequentially or multiple NNs simultaneously—incurring a little bit of loss at each interval. “Unlike space, time is a unidirectional variable. Errors compound with more and more time steps. That’s why we have to think of temporal variables differently from how we approach spatial variables,” explains staff scientist Pratanu Roy, who wrote “Exact Enforcement of Temporal Continuity in Sequential Physics-Informed Neural Networks” with staff scientist Stephen Castonguay. Their paper will appear in the October issue of Computer Methods in Applied Mechanics and Engineering.

Researchers attempt to minimize PINN loss errors by enforcing constraints on boundary conditions, but this “soft” method only approximates continuity between time steps. Instead, the Livermore team developed a “hard” method to exactly enforce continuity between successive time steps. This hard-constrained sequential (HCS) PINN technique leverages optimization algorithms at each time step until convergence is reached. When tested on several benchmark problems, Roy points out, “Our HCS-PINN method outperforms traditional and soft-constrained PINNs in both convergence and accuracy. Our proposed method improves the robustness of PINNs while maintaining the desirable properties of neural network approximators. We think this is an important contribution to scientific ML as this shows a way to solve time-dependent problems for long duration without losing accuracy.”

Image at left: Comparison of hard- [left] and soft- [right] constrained sequential PINN methods for the Korteweg–de Vries PDE, which describes waves on shallow water surfaces. The LLNL team’s method achieves higher accuracy across the same number of time steps. In other benchmarks, they were able to reach an accurate solution in fewer time steps.

To keep employees abreast of the latest AI/ML tools, two data science–focused projects are under way as part of the Lab’s Institutional Scientific Capability Portfolio (ISCP). Distinct from traditional scientific research, ISCP projects improve the Lab’s overall ability to conduct research by overcoming technical and operational hurdles. A new article published on the DSI website describes two successful ISCP projects and their ties to data science.

Shared Education in Artificial intelligence and Machine learning (SEAM) is a professional development program designed to equip employees with job-relevant skills using AI/ML tools. “We’re at a unique moment where there’s high demand for AI and ML expertise, but many new hires may not have this expertise because their academic field hasn’t fully adopted those technologies into their degree programs. Thankfully, the Lab is a place where people want to upskill,” says SEAM program lead Andrew Gillette. (The first cohort of SEAM participants is pictured at left.)

Headed by Brian Weston, the Cloud Services for Mission Science (CSMS) project is designed to help LLNL integrate cloud-based workflows to enhance mission deliverability. “By leveraging tech industry innovations, we can significantly boost our research capabilities and throughput to reduce time-to-discovery,” Weston explains. “Science-enabling technologies such as databases, search engines, Jupyter notebooks, project management tools, and dashboards significantly enhance our overall productivity. These tools continue to develop, especially in the realm of machine learning operations and implementations of large language models.”

A team of scientists at LLNL has developed an ML model to gain an atomic-level understanding of CO₂ capture in amine-based sorbents. This innovative approach promises to enhance the efficiency of direct air capture (DAC) technologies, which are crucial for reducing the excessive amounts of CO₂ already present in the atmosphere. The low cost of these sorbents has enabled several companies to scale up this technology, demonstrating DAC as a feasible way to combat global warming. However, significant knowledge gaps remain in the chemistry of CO₂ capture under experimentally relevant conditions.

The team’s ML model has revealed that CO₂ capture by amines involves the formation of a carbon-nitrogen chemical bond between the amino group and CO₂, alongside a complex set of solvent-mediated proton transfer reactions. These proton transfer reactions are critical for the formation of the most stable CO₂-bound species and are significantly influenced by quantum fluctuations of protons. “By integrating ML with advanced simulation techniques, we’ve created a powerful approach that bridges theoretical predictions and experimental validations of CO₂-capture mechanisms in a way not accessible by traditional simulation techniques,” says Sichi Li, co-corresponding author and project theory lead. The team’s paper appears in Chemical Science.

A recent episode of the “Eye on AI” podcast delves into the cutting-edge world of AI and high-performance computing with Brian Spears, director of LLNL’s AI Innovation Incubator. The episode linked below as a video with the following description: “Brian shares his experience in driving AI into national security science and managing the nation’s nuclear stockpile. With a PhD in mechanical engineering, his expertise spans nonlinear dynamical systems and high-dimensional topology, making him uniquely positioned to lead groundbreaking projects in fusion ignition and AI integration. Discover how LLNL achieved fusion ignition for the first time, harnessing the power of AI to elevate simulation models with precise experimental data. Brian explains how this approach is paving the way for commercially viable fusion energy and advancing stockpile stewardship. Explore the relationship between high-performance computing and AI as Brian discusses the Department of Energy’s FASST initiative. Brian also touches on the importance of public-private partnerships, ethical considerations in AI development, and the future potential of quantum computing. Tune in to understand how the U.S. is leading the global race in AI and computing technology, setting the stage for unprecedented advancements in science and security.” The video runs 56:49; chapters are linked on the YouTube page.

Luc Peterson has applied data science to inertial confinement fusion, high-performance computing, pandemic response, and even space science and security. And he wouldn’t have it any other way: “I’ve always gravitated toward the kind of challenging and captivating, mission-critical problems that stick with you and make you lose track of time,” he says. Today, Peterson is the Associate Program Leader for Data Science in Global Security’s Space Science and Security Program—a far cry from his one-time vision of becoming an attorney. However, an undergraduate research experience at Livermore steered Peterson toward physics, and his passion for the subject culminated in a PhD in plasma physics from Princeton, where he modeled turbulence in fusion reactor experiments at the Princeton Plasma Physics Laboratory. Peterson has now spent over a decade at LLNL working on fascinating, fast-growing technologies. He currently leads an effort preparing for Livermore’s incoming supercomputer, El Capitan, to automate design optimization of National Ignition Facility experiments and demonstrate the potential of advanced simulation and ML at the exascale. He notes, “Data science right now feels similar to the dawn of the computer revolution. We’re graduating from niche applications to new tools and techniques that could transform every aspect of our lives.”