Identification of Nonlinear Conservation Laws Using Symbolic Neural Networks

Abstract

Nonlinear dynamical systems are omnipresent in nature, commonly seen in many disciplines such as physics, biology, chemistry, climate science, and engineering. In this thesis, we introduce several new ideas by integrating machine learning and numerical methods, effectively tackling challenging forward and inverse problems of physical systems. According to [1], research approaches in this field can be broadly categorized based on the synergy between deep learning and domain knowledge into three groups: Supervised Methods, Physics-informed Methods, and Interleaved Methods. Supervised Methods are the classic learning approaches where a physical system produces the data, but no further interaction exists between this physical system and deep learning. In Physics-informed Methods, the physical dynamics are encoded in the loss function, typically in the form of differentiable operations. Interleaved Methods tightly integrate the physical system with the learning process, merging full simulations with deep neural network outputs. Whether addressing Ordinary Differential Equations (ODEs) or Partial Differential Equations (PDEs) in this thesis, the core of our approaches intricately unites Symbolic Neural Networks with ODE & PDE solvers, categorizing our techniques as Interleaved Methods. We start with a slightly simpler task of trying to identify unknown ODEs with parameters from trajectory data in Paper I. Instead of directly learning ODEs using an Ordinary Neural Network (O-Net) [2], we present a novel strategy by combining Symbolic Neural Network (S-Net)—endowed with the capacity to grasp analytical expressions—with an ODE Solver to predict the dynamical system. Our numerical experiments demonstrate that our approach outperforms O-Net when applied to the Lotka-Volterra and Lorenz equations. This discovery concerning the realm of ODEs also imparts valuable insights for our future endeavors in tackling PDEs-related challenges. Graph Neural Networks (GNNs) belonging to Supervised Methods have gained significant attention in recent years due to their ability to process and analyze data structured as graphs. By discretizing continuous spatial domains into grids or meshes, individual grid points or mesh elements can be regarded as nodes within a graph. The connections between nodes can represent the spatial relationships between these points or elements. GNNs have been widely used to solve spatially-dependent PDEs with smooth solutions. In paper II, we explore the application of GNNs to solve conservation laws with non-smooth solutions. Experimental results show that the model can predict accurately when parameters are within a specific range. However, when parameters deviate too much from that used for the training model, the model’s predictive power is significantly reduced. The achievements of the S-Net and ODE Solver in tackling ODEs, coupled with the limitations of GNNs in extending to conservation law challenges, reinforce the need to first learn the expressions of the unknown flux functions of the involved conservation law. From this foundation, we can proceed to forecast subsequent states of the nonlinear dynamical system with greater assurance. In Paper III, we introduce ConsLaw-Net, a combination of S-Net and an entropy-satisfying discretization scheme. This work addresses the problems of one-dimensional conservation law without parameters, and empirical outcomes robustly affirm the effectiveness of our approach. However, the case with conservation laws that involve a parameter, requires that the role of the parameter must also be learned. We propose an appropriate extending to deal robustly with this situation. A two-step learning method, i.e., combining Conslaw-Net and Linear Regression Neural Network (LRNN), is proposed in Paper IV. We test it on two different systems and achieve good results. Furthermore, in paper V, we train the enhanced ConsLaw-Net through a combination of joint and alternating equation strategies, effectively addressing intricate two-dimensional conservation law scenarios demanding high precision despite limited informative data. Conclusively, in Paper VI, we unveil an upgraded ConsLaw-Net tailored for deducing the functional expressions of both flux and diffusion functions within the setting of degenerate convect on-diffusion models, accommodating diverse observation modalities. Taken together, the methodologies outlined in this thesis provide new and hopefully useful tools in the search for hidden nonlinear conservation laws behind a given set of observation data. If we should formulate the main findings of this thesis in a few sentences, it might be as follows: To uncover a possible unknown nonlinear scalar conservation law from synthetic observation data seems attainable by combining appropriate regularity imposed on the unknown function(s), as expressed by the Symbolic Neural Networks, with a "suitable" set of observation data. The meaning of "suitable" here is that small amounts of data might not be sufficient to identify the unknown flux function(s). However, by adding more observation data the proposed method is more and more likely to find the ground truth flux function.