English
Related papers

Related papers: Estimating Model Performance on External Samples f…

200 papers

The impact of machine learning models on healthcare will depend on the degree of trust that healthcare professionals place in the predictions made by these models. In this paper, we present a method to provide people with clinical expertise…

Machine Learning · Computer Science 2021-03-05 Aniruddh Raghu , John Guttag , Katherine Young , Eugene Pomerantsev , Adrian V. Dalca , Collin M. Stultz

We consider the challenges associated with causal inference in settings where data from a randomized trial is augmented with control data from an external source to improve efficiency in estimating the average treatment effect (ATE).…

Clinical researchers often select among and evaluate risk prediction models using standard machine learning metrics based on confusion matrices. However, if these models are used to allocate interventions to patients, standard metrics…

Machine Learning · Statistics 2020-06-03 Alejandro Schuler , Aashish Bhardwaj , Vincent Liu

The information diffusion prediction on social networks aims to predict future recipients of a message, with practical applications in marketing and social media. While different prediction models all claim to perform well, general…

Social and Information Networks · Computer Science 2025-01-16 Wenjin Xie , Xiaomeng Wang , Radosław Michalski , Tao Jia

Machine learning models are typically deployed in a test setting that differs from the training setting, potentially leading to decreased model performance because of domain shift. If we could estimate the performance that a pre-trained…

Computer Vision and Pattern Recognition · Computer Science 2022-07-21 Zeju Li , Konstantinos Kamnitsas , Mobarakol Islam , Chen Chen , Ben Glocker

In certain privacy-sensitive scenarios within fields such as clinical trial simulations, federated learning, and distributed learning, researchers often face the challenge of estimating correlations between variables without access to…

Methodology · Statistics 2025-08-05 Longwen Shang , Min Tsao , Xuekui Zhang

A novel approach to improve prediction and inference in M-estimation by integrating external information from heterogeneous populations is proposed. Our method leverages joint asymptotics to combine estimates from external and internal…

Methodology · Statistics 2025-09-08 Walter Dempsey , Jeremy M. G. Taylor

In many medical and business applications, researchers are interested in estimating individualized treatment effects using data from a randomized experiment. For example in medical applications, doctors learn the treatment effects from…

Methodology · Statistics 2022-03-01 Kevin Wu Han , Han Wu

Multinomial prediction models (MPMs) have a range of potential applications across healthcare where the primary outcome of interest has multiple nominal or ordinal categories. However, the application of MPMs is scarce, which may be due to…

Indirect experiments provide a valuable framework for estimating treatment effects in situations where conducting randomized control trials (RCTs) is impractical or unethical. Unlike RCTs, indirect experiments estimate treatment effects by…

Machine Learning · Computer Science 2023-12-06 Yash Chandak , Shiv Shankar , Vasilis Syrgkanis , Emma Brunskill

Clinical prediction models are estimated using a sample of limited size from the target population, leading to uncertainty in predictions, even when the model is correctly specified. Generally, not all patient profiles are observed…

Methodology · Statistics 2024-02-09 Doranne Thomassen , Saskia le Cessie , Hans van Houwelingen , Ewout Steyerberg

In performative prediction, predictions guide decision-making and hence can influence the distribution of future data. To date, work on performative prediction has focused on finding performatively stable models, which are the fixed points…

Machine Learning · Computer Science 2021-06-17 John Miller , Juan C. Perdomo , Tijana Zrnic

Combining information from multiple samples is often needed in biomedical and economic studies, but the differences between these samples must be appropriately taken into account in the analysis of the combined data. We study estimation for…

Methodology · Statistics 2018-08-14 Heng Shu , Zhiqiang Tan

Integrating information from multiple data sources can enable more precise, timely, and generalizable decisions. However, it is challenging to make valid causal inferences using observational data from multiple data sources. For example, in…

Methodology · Statistics 2023-02-08 Larry Han , Yige Li , Bijan A. Niknam , Jose R. Zubizarreta

The panel data regression models have gained increasing attention in different areas of research including but not limited to econometrics, environmental sciences, epidemiology, behavioral and social sciences. However, the presence of…

Methodology · Statistics 2020-11-24 Beste Hamiye Beyaztas , Soutir Bandyopadhyay

In recent years, there has been a growing interest in the prediction of individualized treatment effects. While there is a rapidly growing literature on the development of such models, there is little literature on the evaluation of their…

Methodology · Statistics 2023-12-22 J Hoogland , O Efthimiou , TL Nguyen , TPA Debray

There is increasing interest in combining information from experimental studies, including randomized and single-group trials, with information from external experimental or observational data sources. Such efforts are usually motivated by…

Methodology · Statistics 2026-04-23 Lawson Ung , Guanbo Wang , Sebastien Haneuse , Miguel A. Hernán , Issa J. Dahabreh

Background: Clinical prediction models are increasingly used to inform healthcare decisions, but determining the minimum sample size for their development remains a critical and unresolved challenge. Inadequate sample sizes can lead to…

Machine Learning · Computer Science 2026-03-02 Diana Shamsutdinova , Felix Zimmer , Oyebayo Ridwan Olaniran , Sarah Markham , Daniel Stahl , Gordon Forbes , Ewan Carr

One approach for increasing the efficiency of randomized trials is the use of "external controls" -- individuals who received the control treatment studied in the trial during routine practice or in prior experimental studies. Existing…

Predicting risks of chronic diseases has become increasingly important in clinical practice. When a prediction model is developed in a given source cohort, there is often a great interest to apply the model to other cohorts. However, due to…

Methodology · Statistics 2020-03-05 Zheng Jiayin , Zheng Yingye , Hsu Li