Our research on ML and X-AI focuses on developing interpretable techniques for modeling liquefaction and full-wave form inversions. We also develop Graph-Network Simulator (GNS) to model large-deformation flows.


  • Graph Network Simulator (GNS)GitHub


GNS simulation of granular flow


X-AI and Machine Learning, 2022 SimCenter-DesignSafe AI workshop


Krishna Kumar

Assistant Professor, UT Austin

Yongjin Choi

PhD candidate, UT Austin

Jodie Crocker

PhD candidate, UT Austin

Cheng-Hsi Hsiao

PhD candidate, UT Austin

Anusha Vajapeyajula

PhD candidate, UT Austin

Joseph Vantassel

PhD, UT Austin


GNS: A generalizable Graph Neural Network-based simulator for particulate and fluid modeling

Kumar, K., Vantassel, J.;

We develop a PyTorch-based Graph Network Simulator (GNS) that learns physics and predicts the flow behavior of particulate and fluid systems. GNS discretizes the domain with nodes representing a collection of material points and the links connecting the nodes representing the local interaction between particles or clusters of particles. The GNS learns the interaction laws through message passing on the graph. GNS has three components: (a) Encoder, which embeds particle information to a latent graph, the edges are learned functions; (b) Processor, which allows data propagation and computes the nodal interactions across steps; and (c) Decoder, which extracts the relevant dynamics (e.g., particle acceleration) from the graph. We introduce physics-inspired simple inductive biases, such as an inertial frame that allows learning algorithms to prioritize one solution (constant gravitational acceleration) over another, reducing learning time. The GNS implementation uses semi-implicit Euler integration to update the next state based on the predicted accelerations. GNS trained on trajectory data is generalizable to predict particle kinematics in complex boundary conditions not seen during training. The trained model accurately predicts within a 5% error of its associated material point method (MPM) simulation. The predictions are 5,000x faster than traditional MPM simulations (2.5 hours for MPM simulations versus 20 s for GNS simulation of granular flow). GNS surrogates are popular for solving optimization, control, critical-region prediction for in situ viz, and inverse-type problems. The GNS code is available under the open-source MIT license at https://github.com/geoelements/gns.

Using explainability to design physics-aware CNNs for solving subsurface inverse problems

Crocker, J., Kumar, K., Cox, B.;

We present a novel method of using explainability techniques to design physics-aware neural networks. We demonstrate our approach by developing a convolutional neural network (CNN) for solving an inverse problem for shallow subsurface imaging. Although CNNs have gained popularity in recent years across many fields, the development of CNNs remains an art, as there are no clear guidelines regarding the selection of hyperparameters that will yield the best network. While optimization algorithms may be used to select hyperparameters automatically, these methods focus on developing networks with high predictive accuracy while disregarding model explainability (descriptive accuracy). However, the field of Explainable Artificial Intelligence (XAI) addresses the absence of model explainability by providing tools that allow developers to evaluate the internal logic of neural networks. In this study, we use the explainability methods Score-CAM and Deep SHAP to select hyperparameters, such as kernel sizes and network depth, to develop a physics-aware CNN for shallow subsurface imaging. We begin with a relatively deep Encoder-Decoder network, which uses surface wave dispersion images as inputs and generates 2D shear wave velocity subsurface images as outputs. Through model explanations, we ultimately find that a shallow CNN using two convolutional layers with an atypical kernel size of 3x1 yields comparable predictive accuracy but with increased descriptive accuracy. We also show that explainability methods can be used to evaluate the network’s complexity and decision-making. We believe this method can be used to develop neural networks with high predictive accuracy while also providing inherent explainability.

Evaluation of particle motions in stabilized specimens of transparent sand using deep learning segmentation

Marx, D., Kumar, K., Zornberg, J.;

Individual particle rotation and displacement were measured in triaxial tests on transparent sand stabilized with geogrid simulants. The Cellpose U-Net model, originally developed to segment biological cells, was trained to segment images of fused quartz particles. The Score-CAM metric from the field of Explainable AI was used to validate the application of Cellpose to segment particles of fused quartz. These segmented particles were characterized in terms of Fourier shape descriptors and tracked across images. The measured particle displacements in the monotonic triaxial tests correlated with displacement fields from Digital Image Correlation (DIC). In contrast to DIC, the new technique also allows for the measurement of individual particle rotation. The particle rotation measurements were found to be repeatable across different specimens. A state boundary line between probable and improbable particle motions could be identified for a given test based on the measured particle displacements and rotations. The size of the zone of probable motions was used to quantify the effectiveness of the stabilizing inclusions. The results of repeated load tests revealed that the honeycomb inclusions used stabilized the specimens by reducing both particle displacements and rotations.

A frequency-velocity CNN for developing near-surface 2D Vs images from linear-array, active-source wavefield measurements

Abbas, A, Vantassel, J., Cox, B. R., Kumar, J., Crocker, J;

This paper presents a frequency-velocity convolutional neural network (CNN) for rapid, non-invasive 2D shear wave velocity (Vs) imaging of near-surface geo-materials. Operating in the frequency-velocity domain allows for significant flexibility in the linear-array, active-source experimental testing configurations used for generating the CNN input, which are normalized dispersion images. Unlike wavefield images, normalized dispersion images are relatively insensitive to the experimental testing configuration, accommodating various source types, source offsets, numbers of receivers, and receiver spacings. We demonstrate the effectiveness of the frequency-velocity CNN by applying it to a classic near-surface geophysics problem, namely, imaging a two-layer, undulating, soil-over-bedrock interface. This problem was recently investigated in our group by developing a time-distance CNN, which showed great promise but lacked flexibility in utilizing different field-testing configurations. Herein, the new frequency-velocity CNN is shown to have comparable accuracy to the time-distance CNN while providing greater flexibility to handle varied field applications. The frequency-velocity CNN was trained, validated, and tested using 100,000 synthetic near-surface models. The ability of the proposed frequency-velocity CNN to generalize across various acquisition configurations is first tested using synthetic near-surface models with different acquisition configurations from that of the training set, and then applied to experimental field data collected at the Hornsby Bend site in Austin, Texas, USA. When fully developed for a wider range of geological conditions, the proposed CNN may ultimately be used as a rapid, end-to-end alternative for current pseudo-2D surface wave imaging techniques or to develop starting models for full waveform inversion.

Minority Report: A graph network oracle for in situ visualization

Kumar, K., Navratil, P., Solis, A., Vantassel, J;

In situ visualization techniques are hampered by a lack of foresight: crucial simulation phenomena can be missed due to a poor sampling rate or insufficient detail at critical timesteps. Keeping a human in the loop is impractical, and defining statistical triggers can be difficult. This paper demonstrates the potential for using a machine-learning-based simulation surrogate as an oracle to identify expected critical regions of a large-scale simulation. These critical regions are used to drive the in situ analysis, providing greater data fidelity and analysis resolution with an equivalent I/O budget to a traditional in situ framework. We develop a distributed asynchronous in situ visualization by integrating TACC Galaxy with CB-Geo MPM for material point simulation of granular flows. We employ a PyTorch-based 3D Graph Network Simulator (GNS) trained on granular flow problems as an oracle to predict the dynamics of granular flows. Critical regions of interests are manually tagged in GNS for in situ rendering in MPM.

A machine learning approach to predicting pore pressure response in liquefiable sands under cyclic loading

Choi, Y., Kumar, K.;

Shear stress history controls the pore pressure response in liquefiable soils. The excess pore pressure does not increase under cyclic loading when shear stress amplitude is lower than the peak prior amplitude – the shielding effect. Many sophisticated constitutive models fail to capture the shielding effect observed in the cyclic liquefaction experiments. We develop a data-driven machine learning model based on the LSTM neural network to capture the liquefaction response of soils under cyclic loading. The LSTM model is trained on 12 laboratory cyclic simple shear tests on Nevada sand in loose and dense conditions subjected to different cyclic simple shear loading conditions. The LSTM model features include the relative density of soil and the previous stress history to predict the pore water pressure response. The LSTM model successfully replicates the pore pressure response for three cyclic simple test results considering the shielding and density effects.

Using Convolutional Neural Networks (CNN) to develop starting models for 2D full waveform inversion

Vantassel, J. P.; Kumar, K.; Cox, B. R.;

Non-invasive subsurface imaging using full waveform inversion (FWI) has the potential to fundamen-tally change engineering site characterization by enabling the recovery of high resolution 2D/3D maps ofsubsurface stiffness. Yet, the accuracy of FWI remains quite sensitive to the choice of the initial start-ing model due to the complexity and non-uniqueness of the inverse problem. In response, we presentthe novel application of convolutional neural networks (CNNs) to transform an experimental seismicwavefield acquired using a linear array of surface sensors directly into a robust starting model for 2DFWI. We begin by describing three key steps used for developing the CNN, which include: selectionof a network architecture, development of a suitable training set, and performance of network training.The ability of the trained CNN to predict a suitable starting model for 2D FWI was compared againstother commonly used starting models for a classic near-surface imaging problem; the identification of anundulating, two-layer, soil-bedrock interface. The CNN developed during this study was able to predictcomplex 2D subsurface images of the testing set directly from their seismic wavefields with an averagemean absolute percent error of 6%. When compared to other common approaches, the CNN approachwas able to produce starting models with smaller seismic image and waveform misfits, both before andafter FWI. The ability of the CNN to generalize to subsurface models which were dissimilar to the onesupon which it was trained was assessed using a more complex, three-layered model. While the predictiveability of the CNN was slightly reduced, it was still able to achieve seismic image and waveform misfitscomparable to the other commonly used starting models. This study demonstrates that CNNs have greatpotential as a tool for developing good starting models for FWI, which are critical for producing accurateFWI results.