English
Related papers

Related papers: How much does your data exploration overfit? Contr…

200 papers

Analyzing interaction data provides an opportunity to learn about users, uncover their underlying goals, and create intelligent visualization systems. The first step for intelligent response in visualizations is to enable computers to infer…

Human-Computer Interaction · Computer Science 2020-10-19 Shayan Monadjemi , Roman Garnett , Alvitta Ottley

Consumers interact with firms across multiple devices, browsers, and machines; these interactions are often recorded with different identifiers for the same consumer. The failure to correctly match different identities leads to a fragmented…

Econometrics · Economics 2023-08-28 Tesary Lin , Sanjog Misra

Data-based decisionmaking must account for the manipulation of data by agents who are aware of how decisions are being made and want to affect their allocations. We study a framework in which, due to such manipulation, data becomes less…

Theoretical Economics · Economics 2022-12-29 Alex Frankel , Navin Kartik

An explorative data analysis system should be aware of what the user already knows and what the user wants to know of the data: otherwise the system cannot provide the user with the most informative and useful views of the data. We propose…

Machine Learning · Statistics 2019-01-01 Kai Puolamäki , Emilia Oikarinen , Buse Atli , Andreas Henelius

As artificial intelligence and machine learning tools become more accessible, and scientists face new obstacles to data collection (e.g., rising costs, declining survey response rates), researchers increasingly use predictions from…

Machine Learning · Statistics 2025-12-08 Stephen Salerno , Kentaro Hoffman , Awan Afiaz , Anna Neufeld , Tyler H. McCormick , Jeffrey T. Leek

Overfitting is the bane of data analysts, even when data are plentiful. Formal approaches to understanding this problem focus on statistical inference and generalization of individual analysis procedures. Yet the practice of data analysis…

Machine Learning · Computer Science 2015-09-28 Cynthia Dwork , Vitaly Feldman , Moritz Hardt , Toniann Pitassi , Omer Reingold , Aaron Roth

An agent must try new behaviors to explore and improve. In high-stakes environments, an agent that violates safety constraints may cause harm and must be taken offline, curtailing any future interaction. Imitating old behavior is safe, but…

Artificial Intelligence · Computer Science 2026-04-17 Drew Prinster , Clara Fannjiang , Ji Won Park , Kyunghyun Cho , Anqi Liu , Suchi Saria , Samuel Stanton

A vast array of transformative technologies developed over the past decade has enabled measurement and perturbation at ever increasing scale, yet our understanding of many systems remains limited by experimental capacity. Overcoming this…

Quantitative Methods · Quantitative Biology 2020-12-25 Brian Cleary , Aviv Regev

We propose a framework to analyze and quantify the bias in adaptive data analysis. It generalizes that proposed by Russo and Zou'15, applying to measurements whose moment generating function exists, measurements with a finite $p$-norm, and…

Information Theory · Computer Science 2017-07-18 Jiantao Jiao , Yanjun Han , Tsachy Weissman

Algorithms and technologies are essential tools that pervade all aspects of our daily lives. In the last decades, health care research benefited from new computer-based recruiting methods, the use of federated architectures for data…

Computers and Society · Computer Science 2023-01-26 Chiara Criscuolo , Tommaso Dolci , Mattia Salnitri

We develop a data-driven information-theoretic framework for sharp partial identification of causal effects under unmeasured confounding. Existing approaches often rely on restrictive assumptions, such as bounded or discrete outcomes;…

Machine Learning · Statistics 2026-02-24 Yonghan Jung , Bogyeong Kang

Learned systems in the domain of visual recognition and cognition impress in part because even though they are trained with datasets many orders of magnitude smaller than the full population of possible images, they exhibit sufficient…

Computer Vision and Pattern Recognition · Computer Science 2022-05-03 John K. Tsotsos , Jun Luo

This paper proposes a unified theoretical model to identify and test a comprehensive set of probabilistic updating biases within a single framework. The model achieves separate identification by focusing on the updating of belief…

General Economics · Economics 2026-03-27 Pedro Gonzalez-Fernandez

It is common in recommendation systems that users both consume and produce information as they make strategic choices under uncertainty. While a social planner would balance "exploration" and "exploitation" using a multi-armed bandit…

Computer Science and Game Theory · Computer Science 2019-02-20 Nicole Immorlica , Jieming Mao , Aleksandrs Slivkins , Zhiwei Steven Wu

Large-scale registries have collected vast amounts of data which has enabled investigators to efficiently conduct studies of observational data. Common practice is for investigators to use all data meeting the inclusion criteria of their…

Statistics Theory · Mathematics 2025-08-25 Reid Dale , Jordan Rodu , Maria E. Currie , Mike Baiocchi

Our consumption of online information is mediated by filtering, ranking, and recommendation algorithms that introduce unintentional biases as they attempt to deliver relevant and engaging content. It has been suggested that our reliance on…

Social and Information Networks · Computer Science 2020-10-07 Dimitar Nikolov , Mounia Lalmas , Alessandro Flammini , Filippo Menczer

Responsible disclosure limitation is an iterative exercise in risk assessment and mitigation. From time to time, as disclosure risks grow and evolve and as data users' needs change, agencies must consider redesigning the disclosure…

We propose information-directed sampling -- a new approach to online optimization problems in which a decision-maker must balance between exploration and exploitation while learning from partial feedback. Each action is sampled in a manner…

Machine Learning · Computer Science 2017-07-10 Daniel Russo , Benjamin Van Roy

Recommender systems must balance personalization, diversity, and robustness to cold-start scenarios to remain effective in dynamic content environments. This paper introduces an adaptive, exploration-based recommendation framework that…

Information Retrieval · Computer Science 2025-03-26 Edoardo Bianchi

Individuals lack oversight over systems that process their data. This can lead to discrimination and hidden biases that are hard to uncover. Recent data protection legislation tries to tackle these issues, but it is inadequate. It does not…

Software Engineering · Computer Science 2023-05-22 Valentin Zieglmeier , Alexander Pretschner