Scientific Machine Learning and Differentiable Simulator

Accurate simulation of granular flow dynamics is crucial for assessing various geotechnical risks, including landslides and debris flows. Granular flows involve a dynamic rearrangement of particles exhibiting complex transitions from solid-like to fluid-like responses. Traditional continuum and discrete numerical methods are limited by their computational cost in simulating large-scale systems. Statistical or machine learning-based models offer an alternative. Still, they are largely empirical, based on a limited set of parameters. Due to their permutation-dependent learning, traditional machine learning-based models require huge training data to generalize. To resolve these problems, we use a graph neural network, a state-of-the-art machine learning architecture that learns local interactions. Graphs represent the state of dynamically changing granular flows and the interaction laws, such as energy and momentum exchange between grains. We develop a graph neural network-based simulator (GNS) that takes the current state of granular flow and predicts the next state using Euler explicit integration by learning the local interaction laws. We train GNS on different granular trajectories. We then assess the performance of GNS by predicting granular column collapse. GNS accurately predicts flow dynamics for column collapses with different aspect ratios unseen during training. GNS is hundreds of times faster than high-fidelity numerical simulators. The model also generalizes to domains much larger than the training data, handling more than twice the number of particles than it was trained on.

Data-driven discoveries require identifying relevant data relationships from a sea of complex, unstructured, and heterogeneous scientific data. We propose a hybrid methodology that extracts metadata and leverages scientific domain knowledge to synthesize a new dataset from the original to construct knowledge graphs. We demonstrate our approach’s effectiveness through a case study on the natural hazard engineering dataset on ``LEAP Liquefaction’’ hosted on DesignSafe. Traditional lexical search on DesignSafe is limited in uncovering hidden relationships within the data. Our knowledge graph enables complex queries and fosters new scientific insights by accurately identifying relevant entities and establishing their relationships within the dataset. This innovative implementation can transform the landscape of data-driven discoveries across various scientific domains.

The widespread adoption of large language models (LLMs), such as OpenAI’s ChatGPT, could revolutionize various industries, including geotechnical engineering. However, GPT models can sometimes generate plausible-sounding but false outputs, leading to hallucinations. In this article, we discuss the importance of prompt engineering in mitigating these risks and harnessing the full potential of GPT for geotechnical applications. We explore the challenges and pitfalls associated with LLMs and highlight the role of context in ensuring accurate and valuable responses. Furthermore, we examine the development of context-specific search engines and the potential of LLMs to become a natural interface for complex tasks, such as data analysis and design. We also develop a unified interface using natural language to handle complex geotechnical engineering tasks and data analysis. By integrating GPT into geotechnical engineering workflows, professionals can streamline their work and develop sustainable and resilient infrastructure systems for the future.

We develop a PyTorch-based Graph Network Simulator (GNS) that learns physics and predicts the flow behavior of particulate and fluid systems. GNS discretizes the domain with nodes representing a collection of material points and the links connecting the nodes representing the local interaction between particles or clusters of particles. The GNS learns the interaction laws through message passing on the graph. GNS has three components: (a) Encoder, which embeds particle information to a latent graph, the edges are learned functions; (b) Processor, which allows data propagation and computes the nodal interactions across steps; and (c) Decoder, which extracts the relevant dynamics (e.g., particle acceleration) from the graph. We introduce physics-inspired simple inductive biases, such as an inertial frame that allows learning algorithms to prioritize one solution (constant gravitational acceleration) over another, reducing learning time. The GNS implementation uses semi-implicit Euler integration to update the next state based on the predicted accelerations. GNS trained on trajectory data is generalizable to predict particle kinematics in complex boundary conditions not seen during training. The trained model accurately predicts within a 5% error of its associated material point method (MPM) simulation. The predictions are 5,000x faster than traditional MPM simulations (2.5 hours for MPM simulations versus 20 s for GNS simulation of granular flow). GNS surrogates are popular for solving optimization, control, critical-region prediction for in situ viz, and inverse-type problems. The GNS code is available under the open-source MIT license at https://github.com/geoelements/gns.

We present a novel method of using explainability techniques to design physics-aware neural networks. We demonstrate our approach by developing a convolutional neural network (CNN) for solving an inverse problem for shallow subsurface imaging. Although CNNs have gained popularity in recent years across many fields, the development of CNNs remains an art, as there are no clear guidelines regarding the selection of hyperparameters that will yield the best network. While optimization algorithms may be used to select hyperparameters automatically, these methods focus on developing networks with high predictive accuracy while disregarding model explainability (descriptive accuracy). However, the field of Explainable Artificial Intelligence (XAI) addresses the absence of model explainability by providing tools that allow developers to evaluate the internal logic of neural networks. In this study, we use the explainability methods Score-CAM and Deep SHAP to select hyperparameters, such as kernel sizes and network depth, to develop a physics-aware CNN for shallow subsurface imaging. We begin with a relatively deep Encoder-Decoder network, which uses surface wave dispersion images as inputs and generates 2D shear wave velocity subsurface images as outputs. Through model explanations, we ultimately find that a shallow CNN using two convolutional layers with an atypical kernel size of 3x1 yields comparable predictive accuracy but with increased descriptive accuracy. We also show that explainability methods can be used to evaluate the network’s complexity and decision-making. We believe this method can be used to develop neural networks with high predictive accuracy while also providing inherent explainability.

Individual particle rotation and displacement were measured in triaxial tests on transparent sand stabilized with geogrid simulants. The Cellpose U-Net model, originally developed to segment biological cells, was trained to segment images of fused quartz particles. The Score-CAM metric from the field of Explainable AI was used to validate the application of Cellpose to segment particles of fused quartz. These segmented particles were characterized in terms of Fourier shape descriptors and tracked across images. The measured particle displacements in the monotonic triaxial tests correlated with displacement fields from Digital Image Correlation (DIC). In contrast to DIC, the new technique also allows for the measurement of individual particle rotation. The particle rotation measurements were found to be repeatable across different specimens. A state boundary line between probable and improbable particle motions could be identified for a given test based on the measured particle displacements and rotations. The size of the zone of probable motions was used to quantify the effectiveness of the stabilizing inclusions. The results of repeated load tests revealed that the honeycomb inclusions used stabilized the specimens by reducing both particle displacements and rotations.

This paper presents a frequency-velocity convolutional neural network (CNN) for rapid, non-invasive 2D shear wave velocity (Vs) imaging of near-surface geo-materials. Operating in the frequency-velocity domain allows for significant flexibility in the linear-array, active-source experimental testing configurations used for generating the CNN input, which are normalized dispersion images. Unlike wavefield images, normalized dispersion images are relatively insensitive to the experimental testing configuration, accommodating various source types, source offsets, numbers of receivers, and receiver spacings. We demonstrate the effectiveness of the frequency-velocity CNN by applying it to a classic near-surface geophysics problem, namely, imaging a two-layer, undulating, soil-over-bedrock interface. This problem was recently investigated in our group by developing a time-distance CNN, which showed great promise but lacked flexibility in utilizing different field-testing configurations. Herein, the new frequency-velocity CNN is shown to have comparable accuracy to the time-distance CNN while providing greater flexibility to handle varied field applications. The frequency-velocity CNN was trained, validated, and tested using 100,000 synthetic near-surface models. The ability of the proposed frequency-velocity CNN to generalize across various acquisition configurations is first tested using synthetic near-surface models with different acquisition configurations from that of the training set, and then applied to experimental field data collected at the Hornsby Bend site in Austin, Texas, USA. When fully developed for a wider range of geological conditions, the proposed CNN may ultimately be used as a rapid, end-to-end alternative for current pseudo-2D surface wave imaging techniques or to develop starting models for full waveform inversion.

In situ visualization techniques are hampered by a lack of foresight: crucial simulation phenomena can be missed due to a poor sampling rate or insufficient detail at critical timesteps. Keeping a human in the loop is impractical, and defining statistical triggers can be difficult. This paper demonstrates the potential for using a machine-learning-based simulation surrogate as an oracle to identify expected critical regions of a large-scale simulation. These critical regions are used to drive the in situ analysis, providing greater data fidelity and analysis resolution with an equivalent I/O budget to a traditional in situ framework. We develop a distributed asynchronous in situ visualization by integrating TACC Galaxy with CB-Geo MPM for material point simulation of granular flows. We employ a PyTorch-based 3D Graph Network Simulator (GNS) trained on granular flow problems as an oracle to predict the dynamics of granular flows. Critical regions of interests are manually tagged in GNS for in situ rendering in MPM.

Shear stress history controls the pore pressure response in liquefiable soils. The excess pore pressure does not increase under cyclic loading when shear stress amplitude is lower than the peak prior amplitude – the shielding effect. Many sophisticated constitutive models fail to capture the shielding effect observed in the cyclic liquefaction experiments. We develop a data-driven machine learning model based on the LSTM neural network to capture the liquefaction response of soils under cyclic loading. The LSTM model is trained on 12 laboratory cyclic simple shear tests on Nevada sand in loose and dense conditions subjected to different cyclic simple shear loading conditions. The LSTM model features include the relative density of soil and the previous stress history to predict the pore water pressure response. The LSTM model successfully replicates the pore pressure response for three cyclic simple test results considering the shielding and density effects.

Non-invasive subsurface imaging using full waveform inversion (FWI) has the potential to fundamen-tally change engineering site characterization by enabling the recovery of high resolution 2D/3D maps ofsubsurface stiffness. Yet, the accuracy of FWI remains quite sensitive to the choice of the initial start-ing model due to the complexity and non-uniqueness of the inverse problem. In response, we presentthe novel application of convolutional neural networks (CNNs) to transform an experimental seismicwavefield acquired using a linear array of surface sensors directly into a robust starting model for 2DFWI. We begin by describing three key steps used for developing the CNN, which include: selectionof a network architecture, development of a suitable training set, and performance of network training.The ability of the trained CNN to predict a suitable starting model for 2D FWI was compared againstother commonly used starting models for a classic near-surface imaging problem; the identification of anundulating, two-layer, soil-bedrock interface. The CNN developed during this study was able to predictcomplex 2D subsurface images of the testing set directly from their seismic wavefields with an averagemean absolute percent error of 6%. When compared to other common approaches, the CNN approachwas able to produce starting models with smaller seismic image and waveform misfits, both before andafter FWI. The ability of the CNN to generalize to subsurface models which were dissimilar to the onesupon which it was trained was assessed using a more complex, three-layered model. While the predictiveability of the CNN was slightly reduced, it was still able to achieve seismic image and waveform misfitscomparable to the other commonly used starting models. This study demonstrates that CNNs have greatpotential as a tool for developing good starting models for FWI, which are critical for producing accurateFWI results.