| dc.description.abstract | In this thesis the Neural Ordinary Differential Equations (NODEs) are studied in their ability to model dynamic systems governed by ODEs. NODEs are a new type of artificial neural network that uses a feed-forward artificial neural network as the source of gradient to construct a continuous trajectory. Although there are several investigations showing NODEs extraordinary ability to model time series, no comprehensive study of the influence of its hyperparameters on its performance has been conducted. In this investigation the objective was to evaluate the influence of some of the NODEs' hyperparameters on the NODEs capabilities of modeling. Special focus was set on the evaluation of the influence of the gradient computation algorithm used, because it determines to a great extent the speed of the training session. Three gradient computation algorithms were analyzed, including a novel method proposed in this thesis; this new approach is based on a modification of the adjoint sensitivity method.
In order to reach these aims, an implementation of NODEs was created using object-oriented programming in the Matlab suite. Then, a group of ODE systems was used to generate several trajectories that were used to train a collection of NODEs that had a different set of hyperparameters. The trained NODEs were used to approximate a set of new trajectories generated by the same systems of ODEs, and the error in the trajectories was used to quantify the influence of the hyperparameters.
The results indicated that the hyperparameters have a big impact on the performance of the NODEs in modeling dynamic systems. Some characteristics of the data to model can give a hint in potential initial hyperparameters, but the evidence showed that many tests need to be done in order to get the optimal hyperparameters. In this regard, the new method proposed for gradient calculation showed potential, because it was ten times faster than the other methods analyzed; that in effect could allow a broader set of hyperparameters to be tested when facing a modeling problem. | |
