Will it bend? Reinforcement learning optimizes metamaterials

Lawrence Livermore staff scientist Xiaoxing Xia collaborated with the Technical University of Denmark to integrate machine learning (ML) and 3D printing techniques. The effort naturally follows Xia’s PhD work in materials science at the California Institute of Technology, where he investigated electrochemically reconfigurable structures. In a paper published in the Journal of Materials Chemistry A, the team detailed their ML approach for devising new shape-changing metamaterials by optimizing their storage capability—an important feature for battery life and other applications.

Metamaterials, Mega Possibilities

Metamaterials generally refer to artificial structures as opposed to those naturally occurring. Xia explains, “Consider how material properties are defined by atomic arrangement in nature: Researchers are limited by materials’ intrinsic crystallographic phase and reactivity properties. An artificial atom, or ‘meta-atom,’ could edit many properties. We can control internal geometry and other parameters of 3D-printed structures to achieve a material with properties that don’t occur naturally. The idea fundamentally challenges our perception of what a material is.”

Incorporating metamaterials could allow lithium-ion batteries to operate more effectively. When a battery is discharged, electrons are stripped from the valence shell of lithium atoms located in the cathode. The electrons travel through the wires of a circuit, producing the current that makes an electrical device function. Meanwhile, inside the battery, the newly ionized lithium atoms migrate away from the cathode and are deposited in the anode—a process known as lithiation. The anode is traditionally composed of graphite, a specific phase of carbon, which accepts the lithium ions into space between the layers of carbon. Next-generation anodes composed of silicon could theoretically store up to ten times the volume of lithium than can the best graphite anodes available today.

Substituting one element for another is no cure-all, however. Silicon does not naturally form the graphite-like layers to house lithium ions. Instead, silicon anodes must be able to withstand lithiation-induced volume expansion to roughly three times their original volume, often causing silicon particles to fracture and thus jeopardizing battery function and longevity. One solution is to build the silicon anode like a building under construction: Horizontal beams of silicon attach to vertical supports to provide space for volume expansion and increase silicon’s ductility.

Xia’s team sought to pin down optimal deformation of the silicon structure to maximize its volume and, by extent, battery performance. They constructed a virtual environment to test structural configurations by compressing the 3D structure into 2D: Vertical supports became nodes, and beams became lines. The model’s generated node placement and the curves drawn between them were kept within a unit square that could be periodically tiled. According to Xia, this simplification was crucial. “My PhD work used complex, fully resolved models that generate accurate simulation results, but doing so is very computationally expensive. This new model significantly reduces the computational complexity so that we can explore a much larger design space.”

Even with extreme graphical simplification, the method used to search for optimal structures would make or break the experiment. With so many nodes to link, an unrefined combinatorial search in this case would have 2²²⁵ configurations to test. “That would be an exhaustive, incredibly inefficient search. Using reinforcement learning [RL] helps us search smarter,” says Xia.

Learning with Less Data

RL differs from other ML paradigms by requiring little to no background data for training and testing. Rather, seeking a user-defined reward, an RL algorithm forges its own way through tasks and weighs whether to seek more information or act off of knowledge already acquired. “This is higher-dimensional learning than humans can visualize, but a computer can detect certain trends and patterns,” Xia explains. With more iterations, the model nears an optimal solution until it reaches a plateau.

The research team chose to reward the model for finding node-and-beam combinations that most efficiently increased storage capacity while avoiding beams overlapping. As volume increased, the beams buckled, bowing out into a ‘C’ shape or a higher-energy, “frustrated” ‘S’ shape depending on whether the nodes could rotate in one direction or two.

Once the model settled on a configuration conducive to maximized storage volume, it was up to Xia to test whether the predictions truly held water—or rather, hexane. “Because they are merely graphs, the results might not translate to reality. My experimental role was to validate the results,” he says. An expert in additive manufacturing technologies, Xia used a stereolithographic printer to reproduce the RL model’s suggestions for initial beam placements. He then flooded the structures with hexane, a liquid hydrocarbon, to simulate the influx of lithium ions during lithiation. He gauged whether structural deformation proceeded as predicted or whether another combination was energetically preferred.

For the non-frustrated scenario, permitting only C-shaped buckling, Xia’s 3D-printed structure behaved as the model predicted. However, in the frustrated scenario supporting C- and S-shaped buckling, the results were more complicated. The printed version agreed with the model’s predictions insofar as exhibiting S-shaped buckling, but node rotations were opposite those expected. Xia says the discrepancy could be due to the model’s necessary oversimplification of a real-world structure: In the printed experiment, the nodes experience deformation and distortion in addition to the beams. This consideration was not built into the computational framework but could be done at the expense a longer and more complicated learning process.

Beyond maximizing battery properties such as energy storage, cycle life, and manufacturability, the RL methods detailed in this study could help devise all sorts of materials with never-before-seen properties. “What makes this approach so widely applicable is that a user could choose any reward and optimize for different properties. Storage capacity was simply the most relevant parameter for us to focus on this time,” he says.

Xia is excited for the future of metamaterials research, including the subset of shape-changing materials. He states, “Shape-changing materials are potentially useful for medical implants, which need to be introduced into the body but then change shape to operate. Also, robotic systems that are too small to fit a motor and gears inside could instead use materials that deform in the presence of electrochemical signals to function. Physical materials themselves might one day even function as analogue of biological neural networks.”