English
Related papers

Related papers: Logistic Regression for Massive Data with Rare Eve…

200 papers

Logistic regression model is widely used in many studies to investigate the relationship between a binary response variable $Y$ and a set of potential predictors $\mathbf X$. The binary response may represent, for example, the occurrence of…

Methodology · Statistics 2021-05-04 Aba Diop , El Hadji Deme , Aliou Diop

Suppose we are given observations, where each observation is drawn independently from one of $k$ known distributions. The goal is to match each observation to the distribution from which it was drawn. We observe that the maximum likelihood…

Data Structures and Algorithms · Computer Science 2019-10-01 Sinho Chewi , Forest Yang , Avishek Ghosh , Abhay Parekh , Kannan Ramchandran

Measurement error in count data is common but underexplored in the literature, particularly in contexts where observed scores are bounded and arise from discrete scoring processes. Motivated by applications in oral reading fluency…

Methodology · Statistics 2025-06-26 Yuqiu Yang , Christina Vu , Cornelis J. Potgieter , Xinlei Wang , Akihito Kamata

By their very nature, rare event probabilities are expensive to compute; they are also delicate to estimate as their value strongly depends on distributional assumptions on the model parameters. Hence, understanding the sensitivity of the…

Numerical Analysis · Mathematics 2021-10-28 Michael Merritt , Alen Alexanderian , Pierre Gremaud

Risk management is particularly concerned with extreme events, but analysing these events is often hindered by the scarcity of data, especially in a multivariate context. This data scarcity complicates risk management efforts. Various tools…

Methodology · Statistics 2026-01-15 Nisrine Madhar , Juliette Legrand , Maud Thomas

Logistic regression is a fundamental and widely used statistical method for modeling binary outcomes based on covariates. However, the presence of missing data, particularly in settings involving hybrid covariates (a mix of discrete and…

Methodology · Statistics 2025-06-05 Mohamed Cherifi , Xujia Zhu , Mohammed Nabil El Korso , Ammar Mesloub

Logistic regression is the most commonly used method for constructing predictive models for binary responses. One significant drawback to this approach, however, is that the asymptotes of the logistic response function are fixed at 0 and 1,…

Methodology · Statistics 2026-02-09 Anthony Almudevar , Jacob Almudevar

Rare events are occurrences that take place with a significantly lower frequency than more common regular events. In manufacturing, predicting such events is particularly important, as they lead to unplanned downtime, shortening equipment…

Machine Learning · Statistics 2024-07-03 Chathurangi Shyalika , Ruwan Wickramarachchi , Fadi El Kalach , Ramy Harik , Amit Sheth

When the difference between treatments in a clinical trial is estimated by a difference in means, then it is well known that randomization ensures unbiassed estimation, even if no account is taken of important baseline covariates. However,…

Statistics Theory · Mathematics 2014-07-22 J. N. S. Matthews , Nuri H. Badi

The case-control sampling design serves as a pivotal strategy in mitigating the imbalanced structure observed in binary data. We consider the estimation of a non-parametric logistic model with the case-control data supplemented by external…

Machine Learning · Statistics 2024-09-04 Hengchao Shi , Ming Zheng , Wen Yu

An approach to the construction of classifiers from imbalanced datasets is described. A dataset is imbalanced if the classification categories are not approximately equally represented. Often real-world data sets are predominately composed…

Artificial Intelligence · Computer Science 2011-11-25 N. V. Chawla , K. W. Bowyer , L. O. Hall , W. P. Kegelmeyer

This paper develops a quasi-maximum likelihood estimator for genuinely unbalanced dynamic network panel data models with individual fixed effects. We propose a model that accommodates contemporaneous and lagged network spillovers, temporal…

Methodology · Statistics 2026-01-01 Zhijian Wang , Xingbai Xu , Tuo Liu

A significant hurdle for analyzing large sample data is the lack of effective statistical computing and inference methods. An emerging powerful approach for analyzing large sample data is subsampling, by which one takes a random subsample…

Methodology · Statistics 2015-11-24 Rong Zhu , Ping Ma , Michael W. Mahoney , Bin Yu

We study nonparametric estimation for current status data with competing risks. Our main interest is in the nonparametric maximum likelihood estimator (MLE), and for comparison we also consider a simpler ``naive estimator.'' Groeneboom,…

Statistics Theory · Mathematics 2008-06-20 Piet Groeneboom , Marloes H. Maathuis , Jon A. Wellner

This study is about inducing classifiers using data that is imbalanced, with a minority class being under-represented in relation to the majority classes. The first section of this research focuses on the main characteristics of data that…

Machine Learning · Computer Science 2022-10-25 Shivaditya Shivganesh , Nitin Narayanan N , Pranav Murali , Ajaykumar M

We study a regression problem where for some part of the data we observe both the label variable ($Y$) and the predictors (${\bf X}$), while for other part of the data only the predictors are given. Such a problem arises, for example, when…

Statistics Theory · Mathematics 2021-04-14 David Azriel , Lawrence D. Brown , Michael Sklar , Richard Berk , Andreas Buja , Linda Zhao

A new family of minimum distance estimators for binary logistic regression models based on $\phi$-divergence measures is introduced. The so called "pseudo minimum phi-divergence estimator"(PM$\phi$E) family is presented as an extension of…

Methodology · Statistics 2016-11-09 Elena Castilla , Nirian Martin , Leandro Pardo

For basic machine learning problems, expected error is used to evaluate model performance. Since the distribution of data is usually unknown, we can make simple hypothesis that the data are sampled independently and identically distributed…

Machine Learning · Computer Science 2022-12-01 Xuli Shen , Qing Xu , Xiangyang Xue

Firth-type logistic regression has become a standard approach for the analysis of binary outcomes with small samples. Whereas it reduces the bias in maximum likelihood estimates of coefficients, bias towards 1/2 is introduced in the…

Methodology · Statistics 2021-01-20 Rainer Puhr , Georg Heinze , Mariana Nold , Lara Lusa , Angelika Geroldinger

Logistic regression is among the most widely used statistical methods for linear discriminant analysis. In many applications, we only observe possibly mislabeled responses. Fitting a conventional logistic regression can then lead to biased…

Applications · Statistics 2017-02-21 Hung Hung , Zhi-Yu Jou , Su-Yun Huang