Related papers: Maximum Likelihood Ridge Regression

Gradient descent, when applied to the task of logistic regression, outputs iterates which are biased to follow a unique ray defined by the data. The direction of this ray is the maximum margin predictor of a maximal linearly separable…

We develop large sample theory including nonparametric confidence regions for $r$-dimensional ridges of probability density functions on $\mathbb{R}^d$, where $1\leq r<d$. We view ridges as the intersections of level sets of some special…

Generative models have been successfully used for generating realistic signals. Because the likelihood function is typically intractable in most of these models, the common practice is to use "implicit" models that avoid likelihood…

This paper studies transfer learning for ridge-regularized robust linear regression in the moderate-dimensional regime, where the number of predictors is of the same order as the sample size and the regression coefficients are not assumed…

Although the foundations of ranking are well established, the ranking literature has primarily been focused on simple, unimodal models, e.g. the Mallows and Plackett-Luce models, that define distributions centered around a single total…

Maximum likelihood estimation (MLE) is a fundamental problem in statistics. Characteristics of the MLE problem for discrete algebraic statistical models are reflected in the geometry of the $\textit{likelihood correspondence}$, a variety…

The Birnbaum-Saunders distribution, also known as the fatigue-life distribution, is frequently used in reliability studies. We obtain adjustments to the Birnbaum--Saunders profile likelihood function. The modified versions of the likelihood…

Penalization of the likelihood by Jeffreys' invariant prior, or by a positive power thereof, is shown to produce finite-valued maximum penalized likelihood estimates in a broad class of binomial generalized linear models. The class of…

This article considers exponential families of truncated multivariate normal distributions with one-sided truncation for some or all coordinates. We observe that if all components are one-sided truncated then this family is not full. The…

We consider settings in which the distribution of a multivariate random variable is partly ambiguous. We assume the ambiguity lies on the level of the dependence structure, and that the marginal distributions are known. Furthermore, a…

Modern regression problems often involve high-dimensional data and a careful tuning of the regularization hyperparameters is crucial to avoid overly complex models that may overfit the training data while guaranteeing desirable properties…

Over the past two decades, shrinkage priors have become increasingly popular, and many proposals can be found in the literature. These priors aim to shrink small effects to zero while maintaining true large effects. Horseshoe-type priors…

Nonparametric regression with random design is considered. Estimates are defined by minimzing a penalized empirical $L_2$ risk over a suitably chosen class of neural networks with one hidden layer via gradient descent. Here, the gradient…

We analyze the problem of maximum likelihood estimation for Gaussian distributions that are multivariate totally positive of order two (MTP2). By exploiting connections to phylogenetics and single-linkage clustering, we give a simple proof…

Being able to predict the occurrence of extreme returns is important in financial risk management. Using the distribution of recurrence intervals---the waiting time between consecutive extremes---we show that these extreme returns are…

The restricted maximum likelihood (REML) estimator of the dispersion matrix for random coefficient models is rewritten in terms of the sufficient statistics of the individual regressions.

Stochastic gradient descent (SGD) provides a simple and efficient way to solve a broad range of machine learning problems. Here, we focus on distribution regression (DR), involving two stages of sampling: Firstly, we regress from…

Suppose that the normal model is used for data $Y_1,\ldots,Y_n$, but that the true distribution is a t-distribution with location and scale parameters $\xi$ and $\sigma$ and $m$ degrees of freedom. The normal model corresponds to…

Reparameterization (RP) and likelihood ratio (LR) gradient estimators are used to estimate gradients of expectations throughout machine learning and reinforcement learning; however, they are usually explained as simple mathematical tricks,…

We present an approximated maximum likelihood method for the multifractal random walk processes of [E. Bacry et al., Phys. Rev. E 64, 026103 (2001)]. The likelihood is computed using a Laplace approximation and a truncation in the…