DSA Thrust Seminar丨Advances of Learning Theory: Statistical Learning Theory for Non-Euclidean Models and General Formula for Information-Theoretic Learning Theory.

Learning theory, which aims to evaluate the performance of machine learning models, is one of the mathematical foundations of machine learning. The evaluation should be conducted on the metric that is suitable for the problem objective. In supervised learning, since the objective is prediction, we use prediction performance as a metric, for which we can use statistical learning theory (SLT). However, previous evaluations by SLT could not apply to non-Euclidean models, which can effectively represent hierarchical data. To overcome this limitation, we have proposed a novel methodology to apply SLT for non-Euclidean models by SLT, clarifying where non-Euclidean models can outperform Euclidean ones. This work will be introduced in the first half of the talk. In unsupervised learning, since the objective is not specific as in supervised learning, we often evaluate a model by information-theoretical learning theory (ITLT) according to minimum description length principle, using a normalized maximum likelihood (NML) code length as a metric. While previous work has derived NML code length calculation formulae independently for a few models, they have not provided any general formula. We have derived a novel general formula of the NML code length, covering the whole exponential family. The formula also shows that the NML code length is directly related to Bayesian statistics through complex function theory. This work will be introduced in the second half of this talk.