Department of Mathematics - Seminar on Applied Mathematics - Regularized Learning in Reproducing Kernel Banach Spaces

Regularized learning is the contemporary framework for learning to generalize from finite samples (classification, regression, clustering, etc). Here the problem is to learn an input-output mapping f: X->Y given finite samples {(xi, yi), i=1,…,N}. The learning-from-data problem is then formulated as an optimization problem in a function space, with the objective function consisting of a loss term L(f) capturing its goodness-of-fit (or the lack thereof) on given samples {(f(xi), yi), i=1,…,N}, and a penalty term R(f) capturing its complexity based on prior knowledge about the solution (smoothness, sparsity, etc). This results in the celebrated Reproducing Kernel Hilbert Space methods in machine learning. Here, we will remove the Hilbert space restriction, i.e., the existence of an inner product, and show that the key ingredients of this framework (reproducing kernel, representer theorem, feature space) remain to hold for a Banach space that is uniformly convex and uniformly Frechet differentiable. Central to our development is the use of a semi-inner product operator and duality mapping for a uniform Banach space in place of an inner-product for a Hilbert space. This opens up the possibility of unifying kernel-based methods (regularizing L2-norm) and sparsity-based methods (regularizing l1-norm), which have so far been investigated under different theoretical foundations. (Joint work with Haizhang Zhang and Yuesheng Xu).