Owen Davis: Mathematically rigorous deep learning paradigms for data driven scientific modeling

Abstract:

Data driven modeling is integral to numerous science and engineering applications, and under this umbrella, a common task is the construction of cheap-to-evaluate data driven surrogate models to accelerate outer- and/or inner-loop tasks associated with large scale computational models. In cases when the computational model is particularly expensive, and we can only afford a handful of forward simulations, we face the challenging task of training a surrogate model from sparse high fidelity data. Under these conditions, it is generally mandatory that the surrogate be constructed from multifidelity training data, sparse high fidelity data together with a set of more plentiful lower fidelity data that is cheaper to obtain but less accurate. Recently, together with growing interest in artificial intelligence, there has been a thrust to develop deep learning paradigms to tackle these difficult scientific data driven modeling problems. While many empirically successful modeling paradigms have by now appeared in the literature, there remains a comparatively underdeveloped body of crosscutting mathematical theory that clarifies why, with what level of certainty, and under what circumstances these methods work. This talk concerns the development of deep learning based modeling paradigms and the mathematical theory that supports their use.

In particular, we focus on what random Fourier features residual networks and ReLU networks can offer us for both single fidelity and multifidelity modeling tasks. In a single fidelity context, we exhibit a global optimization free training strategy with error control for random Fourier features residual networks. We show that this training strategy yields a consistently observed theoretical approximation rate, interpretable learned network parameters, and facilitates efficient approximation of multiscale target function features. We further show that, for a large class of bounded functions with minimal regularity assumptions, the ReLU network approximation error is proportional to the uniform norm of the target function and inversely proportional to the network complexity. In a multifidelity context, we exhibit how this new ReLU approximation error estimate informs multifidelity modeling choices and clarifies under what conditions they are performative. We then develop a bifidelity modeling strategy that leverages one layer random Fourier features networks, and we exhibit its ability to consistently learn interpretable representations of the target function on sparse high fidelity training data.