Convergence of the EM algorithm in KL distance for overspecified Gaussian mixtures
Legg A. Pak A. Melnykov I. Bolatov A. Assylbekov Z.
October 2025Springer Science and Business Media Deutschland GmbH
Statistical Papers
2025#66Issue 6
We present a study of the convergence properties of the Expectation-Maximization (EM) algorithm when applied to an overspecified model. In particular, we consider fitting a balanced mixture of two Gaussians to data originating from a single Gaussian. We provide theoretical bounds on the Kullback–Leibler (KL) divergence between the fitted and true distributions. An important feature is concavity and radiality of the expected log-likelihood function on a hypersurface induced by the EM algorithm, which greatly simplifies the analysis. We also show how our result on KL divergence can be used to upperbound the error rate of a mixture discriminant analysis classifier trained by the EM algorithm.
Expectation-maximization , KL divergence , Mixture models
Text of the article Перейти на текст статьи
Department of Mathematical Sciences, Purdue University Fort Wayne, 2101 East Coliseum Boulevard, Fort Wayne, 46805, IN, United States
Department of Mathematics, Nazarbayev University, 53 Kabanbay Batyr ave., Astana, 010000, Kazakhstan
Department of Mathematics and Statistics, University of Minnesota Duluth, 1049 University Drive, Duluth, 55812, MN, United States
Machine Learning Department, Mohamed bin Zayed University of Artificial Intelligence, Masdar City, Abu Dhabi, 00000, United Arab Emirates
Institute of Mathematics and Mathematical Modeling, 125 Pushkin str, Almaty, 050010, Kazakhstan
Department of Mathematical Sciences
Department of Mathematics
Department of Mathematics and Statistics
Machine Learning Department
Institute of Mathematics and Mathematical Modeling
10 лет помогаем публиковать статьи Международный издатель
Книга Публикация научной статьи Волощук 2026 Book Publication of a scientific article 2026