Department of Mathematics - Seminar on Applied Mathematics - Homotopy Dynamics for Neural Networks in Solving Partial Differential Equations

Solving partial differential equations (PDEs) using neural networks has become a famous topic in scientific machine learning. However, training neural networks remains challenging due to the highly complex and non-convex energy landscapes of the associated loss functions. These difficulties are further amplified in sharp interface problems, where certain parameters in the PDEs introduce near-singularities in the loss. In this talk, I will present a novel training framework based on homotopy dynamics to address these challenges. Specifically, I will introduce two homotopy strategies: the first performs homotopy in the activation functions by gradually transforming from simpler to the original nonlinearities; the second applies homotopy in the PDE parameters to manage the singular behavior in sharp interface regimes. Both approaches demonstrate improved training stability and enhanced accuracy in capturing sharp interfaces when solving PDEs with neural networks.