Related papers: The Foundation of Big Data: Experiments, Formulati…
Big data is data that exceeds the processing capacity of traditional databases. The data is too big to be processed by a single machine. New and innovative methods are required to process and store such large volumes of data. This paper…
In the big data era, the need to reevaluate traditional statistical methods is paramount due to the challenges posed by vast datasets. While larger samples theoretically enhance accuracy and hypothesis testing power without increasing false…
In this paper, we study inference for high-dimensional data characterized by small sample sizes relative to the dimension of the data. In particular, we provide an infinite-dimensional framework to study statistical models that involve…
Data is a critical aspect of the world we live in. With systems producing and consuming vast amounts of data, it is essential for businesses to digitally transform and be equipped to derive the most value out of data. Data analytics…
The concept of complexity appears in virtually all areas of knowledge. Its intuitive meaning shares similarities across fields, but disagreements between its details hinders a general definition, leading to a plethora of proposed…
The era of big data has witnessed an increasing availability of multiple data sources for statistical analyses. We consider estimation of causal effects combining big main data with unmeasured confounders and smaller validation data with…
In this paper we consider some of the issues of working with big data and big spatial data and highlight the need for an open and critical framework. We focus on a set of challenges underlying the collection and analysis of big data. In…
Recent and forthcoming advances in instrumentation, and giant new surveys, are creating astronomical data sets that are not amenable to the methods of analysis familiar to astronomers. Traditional methods are often inadequate not merely…
The massive data sets from today's particle physics experiments present a variety of challenges amenable to the tools developed by the statistics community. From the real-time decision of what subset of data to record on permanent storage,…
This introduction to Bayesian statistics presents the main concepts as well as the principal reasons advocated in favour of a Bayesian modelling. We cover the various approaches to prior determination as well as the basis asymptotic…
The paper argues that a part of the current statistical discussion is not based on the standard firm foundations of the field. Among the examples we consider are prediction into the future, semi-supervised classification, and causality…
Data analysis and data mining are concerned with unsupervised pattern finding and structure determination in data sets. The data sets themselves are explicitly linked as a form of representation to an observational or otherwise empirical…
Statistical physics aims to describe properties of macroscale systems in terms of distributions of their microscale agents. Its central tool is the maximization of entropy, a variational principle. We review the history of this principle,…
Dramatic increases in the size and complexity of modern datasets have made traditional "centralized" statistical inference prohibitive. In addition to computational challenges associated with big data learning, the presence of numerous data…
This paper summarizes a presentation for a panel discussion on "The Future of Astrostatistics" held at the Statistical Challenges in Modern Astronomy V conference at Pennsylvania State University in June 2011. I argue that the emerging…
These notes are intended as an introduction to a study of applications of noncommutative calculus to quantum statistical Physics. Centered on noncommutative calculus we describe the physical concepts and mathematical structures appearing in…
Different viewpoints on the asymptotic expansion of Feynman diagrams are reviewed. The relations between the field theoretic and diagrammatic approaches are sketched. The focus is on problems with large masses or large external momenta.…
When can reliable inference be drawn in the "Big Data" context? This paper presents a framework for answering this fundamental question in the context of correlation mining, with implications for general large scale inference. In large…
Topological statistical theory provides the foundation for a modern mathematical reformulation of classical statistical theory: Structural Statistics emphasizes the structural assumptions that accompany distribution families and the set of…
Superstatistics is a widely employed tool of non-equilibrium statistical physics which plays an important role in analysis of hierarchical complex dynamical systems. Yet, its "canonical" formulation in terms of a single nuisance parameter…