FINTECH THRUST SEMINAR | Nonlinear Monte Carlo and deep neural networks can approximate semilinear partial differential equations without the curse of dimensionality  

Nonlinear Monte Carlo and deep neural networks can approximate semilinear partial differential equations without the curse of dimensionality

Abstract:

Partial differential equations (PDEs) are among the most universal tools used in modeling problems in nature and man-made complex systems. Nearly all traditional approximation algorithms for PDEs in the literature suffer from the so-called "curse of dimensionality" in the sense that the number of required computational operations of the approximation algorithm to achieve a given approximation accuracy grows exponentially in the dimension of the considered PDE. With such algorithms it is impossible to approximately compute solutions of high-dimensional PDEs even when the fastest currently available computers are used. In the specific situation of linear parabolic PDEs and approximations at a fixed space-time point, the curse of dimensionality of deterministic methods can be overcome by means of Monte Carlo approximation algorithms and the Feynman-Kac formula. In this talk we show that deep neural networks (DNNs) have the fundamental property to be able to approximate solutions of semilinear PDEs with Lipschitz nonlinearities with the number of real parameters of the approximating DNN growing at most polynomially in, both, the reciprocal of the prescribed approximation accuracy and the PDE dimension. Our arguments are strongly based on suitable nonlinear Monte Carlo methods for such PDEs. In the second part of the talk we present and analyze acceleration techniques for stochastic gradient descent optimization in the context of deep learning approximations for PDEs and optimal control problems.