Related papers: Resolving the Lord's Paradox
In many applications, particularly in the natural sciences, the available high-dimensional set of features may contain variables that are not correlated with the response under consideration. Such irrelevant features can, in certain cases,…
A connection between the General Linear Model (GLM) in combination with classical statistical inference and the machine learning (MLE)-based inference is described in this paper. Firstly, the estimation of the GLM parameters is expressed as…
I think we can agree that dealing with uncertainty is not easy. Probability is the main tool for dealing with uncertainty, and we know there are many probability-related puzzles and paradoxes. Here I describe a rather idiosyncratic…
How should researchers analyze randomized experiments in which the main outcome is latent and measured in multiple ways but each measure contains some degree of error? We first identify a critical study-specific noncomparability problem in…
We propose a principal components regression method based on maximizing a joint pseudo-likelihood for responses and predictors. Our method uses both responses and predictors to select linear combinations of the predictors relevant for the…
Linear regression is a fundamental modeling tool in statistics and related fields. In this paper, we study an important variant of linear regression in which the predictor-response pairs are partially mismatched. We use an optimization…
Hidden variable graphical models can sometimes imply constraints on the observable distribution that are more complex than simple conditional independence relations. These observable constraints can falsify assumptions of the model that…
This paper analyzes the classical linear regression model with measurement errors in all the variables. First, we provide necessary and sufficient conditions for identification of the coefficients. We show that the coefficients are not…
The main goal of this paper is the study of two kinds of nonlinear problems depending on parameters in unbounded domains. Using a nonstandard variational approach, we first prove the existence of bounded solutions for nonlinear eigenvalue…
Least squares linear regression is one of the oldest and widely used data analysis tools. Although the theoretical analysis of the ordinary least squares (OLS) estimator is as old, several fundamental questions are yet to be answered.…
We investigate the nonlinear regression problem under L2 loss (square loss) functions. Traditional nonlinear regression models often result in non-convex optimization problems with respect to the parameter set. We show that a convex…
By using both, the weak-value formulation as well as the standard probabilistic approach, we analyze the Hardy's experiment introducing a complex and dimensionless parameter ($\epsilon$) which eliminates the assumption of complete…
This paper tackles the challenge of detecting unreliable behavior in regression algorithms, which may arise from intrinsic variability (e.g., aleatoric uncertainty) or modeling errors (e.g., model uncertainty). First, we formally introduce…
In a regression setting with a response vector and given regressor vectors, a typical question is to what extent the response is related to these regressors, specifically, how well it can be approximated by a linear combination of the…
Logistic regression is an important statistical tool for assessing the probability of an outcome based upon some predictive variables. Standard methods can only deal with precisely known data, however many datasets have uncertainties which…
Symbolic regression is the machine learning method for learning functions from data. After a brief overview of the symbolic regression landscape, I will describe the two main challenges that traditional algorithms face: they have an unknown…
Reasoning about unpredicted change consists in explaining observations by events; we propose here an approach for explaining time-stamped observations by surprises, which are simple events consisting in the change of the truth value of a…
It is well known that models used in conventional regression analysis are commonly misspecified. A standard response is little more than a shrug. Data analysts invoke Box's maxim that all models are wrong and then proceed as if the results…
Every observation may follow a distribution that is randomly selected in a class of distributions. It is called the distribution uncertainty. This is a fact acknowledged in some research fields such as financial risk measure. Thus, the…
Probabilistic models often have parameters that can be translated, scaled, permuted, or otherwise transformed without changing the model. These symmetries can lead to strong correlation and multimodality in the posterior distribution over…