Related papers: Convergence of Estimators in LLS Analysis
We study the rates of estimation of finite mixing distributions, that is, the parameters of the mixture. We prove that under some regularity and strong identifiability conditions, around a given mixing distribution with $m_0$ components,…
Probabilistic models help us encode latent structures that both model the data and are ideally also useful for specific downstream tasks. Among these, mixture models and their time-series counterparts, hidden Markov models, identify…
This paper examines the local linear regression (LLR) estimate of the conditional distribution function $F(y|x)$. We derive three uniform convergence results: the uniform bias expansion, the uniform convergence rate, and the uniform…
M-estimators for Generalized Linear Models are considered under minimal assumptions. Under these preliminaries, strong convergence of the estimators are discussed and an expansion of the estimating operators are given in the non-i.i.d. case…
Linear mixed models (LMMs) are suitable for clustered data and are common in biometrics, medicine, survey statistics and many other fields. In those applications, it is essential to carry out valid inference after selecting a subset of the…
Rigorous evaluation of large language models (LLMs) relies on comparing models by the prevalence of desirable or undesirable behaviors, such as task pass rates or policy violations. These prevalence estimates are produced by a classifier,…
The Box-Cox transformation is applied to the linear mixed models for analyzing positive and grouped data. The problem in using Box Cox transformation is that the maximum likelihood estimator of the transformation parameter is generally…
We investigate the in-context learning capabilities of transformers for the $d$-dimensional mixture of linear regression model, providing theoretical insights into their existence, generalization bounds, and training dynamics. Specifically,…
Two moment-matching methods for model reduction of linear switched systems (LSSs) are presented. The methods are similar to the Krylov subspace methods used for moment matching for linear systems. The more general one of the two methods, is…
Maronna's and Tyler's $M$-estimators are among the most widely used robust estimators for scatter matrices. However, when the dimension of observations is relatively high, their performance can substantially deteriorate in certain…
In this paper, we construct a moment inequality for mixing dependent random variables, it is of independent interest. As applications, the consistency of the kernel density estimation is investigated. Several limit theorems are established:…
Testing whether a probability distribution is compatible with a given Bayesian network is a fundamental task in the field of causal inference, where Bayesian networks model causal relations. Here we consider the class of causal structures…
Mixture distributions arise in many parametric and non-parametric settings -- for example, in Gaussian mixture models and in non-parametric estimation. It is often necessary to compute the entropy of a mixture, but, in most cases, this…
This paper deals with the state estimation of linear time-invariant systems using distributed observers with local sampled-data measurement and aperiodic communication. Each observer agent perceives partial information of the system to be…
Finite mixture models are widely used in econometric analyses to capture unobserved heterogeneity. This paper shows that maximum likelihood estimation of finite mixtures of parametric densities can suffer from substantial finite-sample bias…
We consider the problem of joint estimation of structured inverse covariance matrices. We perform the estimation using groups of measurements with different covariances of the same unknown structure. Assuming the inverse covariances to span…
The condition of parameter identifiability is essential for the consistency of all estimators and is often challenging to prove. As a consequence, this condition is often assumed for simplicity although this may not be straightforward to…
We present a new finite-time analysis of the estimation error of the Ordinary Least Squares (OLS) estimator for stable linear time-invariant systems. We characterize the number of observed samples (the length of the observed trajectory)…
The article introduces spatial long-range dependent models based on the fractional difference operators associated with the Gegenbauer polynomials. The results on consistency and asymptotic normality of a class of minimum contrast…
Gradient information on the sampling distribution can be used to reduce the variance of Monte Carlo estimators via Stein's method. An important application is that of estimating an expectation of a test function along the sample path of a…