English
Related papers

Related papers: Characterizing Evolution in Expectation-Maximizati…

200 papers

We study the convergence of the Expectation-Maximization (EM) algorithm for mixtures of linear regressions with an arbitrary number $k$ of components. We show that as long as signal-to-noise ratio (SNR) is $\tilde{\Omega}(k)$,…

Machine Learning · Computer Science 2019-11-27 Jeongyeol Kwon , Constantine Caramanis

While mixture of linear regressions (MLR) is a well-studied topic, prior works usually do not analyze such models for prediction error. In fact, {\em prediction} and {\em loss} are not well-defined in the context of mixtures. In this paper,…

Machine Learning · Statistics 2022-05-27 Avishek Ghosh , Arya Mazumdar , Soumyabrata Pal , Rajat Sen

Emergent misalignment (EM), where fine-tuning on a narrow task (like insecure code) causes broad misalignment across unrelated domains, was first demonstrated by Betley et al. (2025). We conduct the most comprehensive EM study to date,…

Machine Learning · Computer Science 2026-05-13 Joel Schreiber , Ariel Goldstein

We study the expectation-maximization (EM) algorithm for general latent-variable models under (i) distributional misspecification and (ii) nonidentifiability induced by a group action. We formulate EM on the quotient parameter space and…

Statistics Theory · Mathematics 2026-01-06 Koustav Mallik

Mixtures-of-Experts (MoE) are conditional mixture models that have shown their performance in modeling heterogeneity in data in many statistical learning approaches for prediction, including regression and classification, as well as for…

Methodology · Statistics 2019-07-17 Bao Tuyen Huynh , Faicel Chamroukhi

The problem of linear predictions has been extensively studied for the past century under pretty generalized frameworks. Recent advances in the robust statistics literature allow us to analyze robust versions of classical linear models…

Machine Learning · Statistics 2022-03-15 Saptarshi Chakraborty , Debolina Paul , Swagatam Das

We consider a semiparametric mixture of two univariate density functions where one of them is known while the weight and the other function are unknown. Such mixtures have a history of application to the problem of detecting differentially…

Statistics Theory · Mathematics 2017-08-01 Zhou Shen , Michael Levine , Zuofeng Shang

Mixed linear regression (MLR) is a powerful model for characterizing nonlinear relationships by utilizing a mixture of linear regression sub-models. The identification of MLR is a fundamental problem, where most of the existing results…

Machine Learning · Statistics 2023-12-01 Yujing Liu , Zhixin Liu , Lei Guo

Mixture models serve as one fundamental tool with versatile applications. However, their training techniques, like the popular Expectation Maximization (EM) algorithm, are notoriously sensitive to parameter initialization and often suffer…

Machine Learning · Computer Science 2023-12-20 Yulai Cong , Sijia Li

In fitting a mixture of linear regression models, normal assumption is traditionally used to model the error and then regression parameters are estimated by the maximum likelihood estimators (MLE). This procedure is not valid if the normal…

Methodology · Statistics 2018-11-06 Yanyuan Ma , Shaoli Wang , Lin Xu , Weixin Yao

Mixture modeling, which considers the potential heterogeneity in data, is widely adopted for classification and clustering problems. Mixture models can be estimated using the Expectation-Maximization algorithm, which works with the complete…

Methodology · Statistics 2022-03-18 Shonosuke Sugasawa , Genya Kobayashi

In this paper, different strands of literature are combined in order to obtain algorithms for semi-parametric estimation of discrete choice models that include the modelling of unobserved heterogeneity by using mixing distributions for the…

Methodology · Statistics 2022-12-12 Dietmar Bauer , Sebastian Büscher , Manuel Batram

This paper presents three hyperspectral mixture models jointly with Bayesian algorithms for supervised hyperspectral unmixing. Based on the residual component analysis model, the proposed general formulation assumes the linear model to be…

Data Analysis, Statistics and Probability · Physics 2016-08-24 Abderrahim Halimi , Paul Honeine , Jose Bioucas-Dias

Mixture of linear regression is well studied in statistics and machine learning, where the data points are generated probabilistically using $k$ linear models. Algorithms like Expectation Maximization (EM) may be used to recover the ground…

Machine Learning · Computer Science 2026-04-08 Avishek Ghosh

Finite mixture models have been widely used to model and analyze data from a heterogeneous populations. Moreover, data of this kind can be missing or subject to some upper and/or lower detection limits because of the restriction of…

We study behavior of the restricted maximum likelihood (REML) estimator under a misspecified linear mixed model (LMM) that has received much attention in recent gnome-wide association studies. The asymptotic analysis establishes consistency…

Statistics Theory · Mathematics 2014-04-10 Jiming Jiang , Cong Li , Debashis Paul , Can Yang , Hongyu Zhao

The Expectation-Maximization (EM) algorithm is a widely used method for maximum likelihood estimation in models with latent variables. For estimating mixtures of Gaussians, its iteration can be viewed as a soft version of the k-means…

Machine Learning · Statistics 2017-06-06 Constantinos Daskalakis , Christos Tzamos , Manolis Zampetakis

We consider the problem of learning a model from multiple heterogeneous sources with the goal of performing well on a new target distribution. The goal of learner is to mix these data sources in a target-distribution aware way and…

Machine Learning · Computer Science 2023-11-14 Yuyang Deng , Ilja Kuzborskij , Mehrdad Mahdavi

Machine learning (ML) models are typically optimized for their accuracy on a given dataset. However, this predictive criterion rarely captures all desirable properties of a model, in particular how well it matches a domain expert's…

Machine Learning · Computer Science 2022-07-07 Damien Teney , Maxime Peyrard , Ehsan Abbasnejad

Improved EM strategies, based on the idea of efficient data augmentation (Meng and van Dyk 1997, 1998), are presented for ML estimation of mixture proportions. The resulting algorithms inherit the simplicity, ease of implementation, and…

Computation · Statistics 2010-02-22 Yaming Yu