Reinforcement Learning Optimizer for Earthquakes Simulations using Fiber Bundle Models
Presenter
DescriptionRupture of any heterogeneous material is a complex physical process difficult to model deterministic due to the number of unmeasurable parameters involved and the poorly constrained physical conditions. The lack of long seismic series, due to our short instrumental recording time, makes it difficult to observe whole seismic cycles. Thus, the predictive potential of these phenomena usually becomes insufficient. One of the main goals is to explore new approaches able to generate accurate synthetic time series (physically and statistically) aiming to produce a better understanding of the earthquake phenomenon. In this sense, an earthquake simulator based on the Fiber Bundle Model (FBM) that produces synthetic series fulfilling seismic statistical patterns has been recently developed, in particular, those series related to the mainshock and the aftershock sequences (Monterrubio-Velasco et al. 2019a, 2019b, 2020). This new model has been coined as TREMOL (sTochastic Rupture Earthquake MOdeL). The FBM is a model whose algorithm is based upon the interaction of individual elements, with particular charge transfer rules and a probability distribution function to describe the intrinsic properties of its constituent elements. This model offers many advantages and great adaptability to describe various rupture phenomena, from the modeling of rupture in microscopic composite materials to large-scale rupture phenomena such as earthquakes. One of the most important features of TREMOL is that it requires a deep parameter tuning that can significantly improve the approximation of the synthetic results with respect to the real ones. The correct parameterization of TREMOL generates seismic synthetic catalogs consistent with those observed in nature, thus adjusting the most important empirical relationships of seismology. Unfortunately the strong stochastic and discrete nature of the FBM hinders the application of classical optimization techniques based on, for example, continuous gradient descent methods. As a promising alternative to these approaches, supervised machine learning (ML) classification algorithms have been recently used to predict the best parameter values associated with some preselected classes (Monterrubio et al., 2018, Llácer et al., 2020). Those algorithms demonstrate high performance in solving this problem for the specific aftershock application, producing a synthetic behavior of earthquakes close enough to the observed one. However, note that this optimization strategy is inherently discrete due to the ML classification, requiring in general costly training and producing less accurate results as long as the number of classes increases. More precisely, the explored supervised techniques were applied to analyze three parameters requiring a large amount of pre-executed simulations to train the ML models. In cases where the model complexity increases (i.e. increasing the dimensionality by adding more features, classes, and spatial dimensions) the previous approach may be computationally unaffordable for optimizing the TREMOL model. Trying to overcome some of the aforementioned drawbacks, in this work we explore another alternative strategy to optimize the TREMOL model following an artificial intelligence (AI) based approach. Instead of performing a supervised method to learn the best parameter class, the key idea is building an artificial agent that learns from its own experience (with no supervision) which is the optimal parameter value that maximizes a given goal function. This AI paradigm is known as reinforcement learning (RL), where the agent interacts with its environment by taking actions and evaluating a reward signal. The final goal is to learn a policy that transforms a current environment state into an action that potentially returns the maximum accumulation of rewards, taking into account all the possibilities. Here, we reformulate the RL paradigm as an optimization problem for the TREMOL environment and build an artificial agent that deals with continuous actions as the values of the FBM parameters. The RL algorithms work naturally in high dimensional spaces and benefit from multiple ways of addressing high-performance implementations, for instance, the possibility of distributing numerous environment instances for those cases where TREMOL requires higher computational costs.
TimeThursday, 8 July 202111:30 - 12:00 CEST
LocationJean Calvin
Session Chair
Event Type
Minisymposium
CS and Math
Emerging Applications
Climate and Weather
Solid Earth Dynamics