Watanabe’s Singular Learning Theory

Welcome. This website contains some notes taken while exploring Sumio Watanabe’s Mathematical Theory of Bayesian Statistics (Singular Learning Theory) as well as simulations, mostly written by Gemini 3 Pro.

Contents

• Cheat Sheet: Important Definitions and Relations
• Regular and Singular Models (Reproducing Figures 1.2 to 1.5)
• Illustrating Watanabe’s Singular Learning Theory with a Gaussian Mixture Model
• Comparing stochastic gradient descent with the posterior distribution
• Neural Network (Reproducing Figure 2.7)