Related papers: Statistical Mechanics of Learning: A Variational A…
Lecture notes from the course given by Professor Sara A. Solla at the Les Houches summer school on "Statistical physics of Machine Learning". The notes discuss neural information processing through the lens of Statistical Physics. Contents…
Scientific and business practices are increasingly resulting in large collections of randomized experiments. Analyzed together, these collections can tell us things that individual experiments in the collection cannot. We study how to learn…
The empirical risk minimization approach to data-driven decision making requires access to training data drawn under the same conditions as those that will be faced when the decision rule is deployed. However, in a number of settings, we…
Despite the development of numerous adaptive optimizers, tuning the learning rate of stochastic gradient methods remains a major roadblock to obtaining good practical performance in machine learning. Rather than changing the learning rate…
The study of the dynamics of the size of a population via mathematical modelling is a problem of interest and widely studied. Traditionally, continuous deterministic methods based on differential equations have been used to deal with this…
A new method called "variational sampling" is proposed to estimate integrals under probability distributions that can be evaluated up to a normalizing constant. The key idea is to fit the target distribution with an exponential family model…
A generalization of Gy's theory for the variance of the fundamental sampling error is reviewed. Practical situations where the generalized model potentially leads to more accurate variance estimates are identified as: clustering of…
Modeling the temporal behavior of data is of primordial importance in many scientific and engineering fields. Baseline methods assume that both the dynamic and observation equations follow linear-Gaussian models. However, there are many…
Learning behavioral patterns from observational data has been a de-facto approach to motion forecasting. Yet, the current paradigm suffers from two shortcomings: brittle under distribution shifts and inefficient for knowledge transfer. In…
Simulation methods are among the most ubiquitous methodological tools in statistical science. In particular, statisticians often is simulation to explore properties of statistical functionals in models for which developed statistical theory…
We present a Bayesian approach to machine learning with probabilistic programs. In our approach, training on available data is implemented as inference on a hierarchical model. The posterior distribution of model parameters is then used to…
We introduce a stochastic fractional calculus. As an application, we present a stochastic fractional calculus of variations, which generalizes the fractional calculus of variations to stochastic processes. A stochastic fractional…
We introduce a few of the key ideas of statistical analysis using two real-world examples to illustrate how these ideas are used in practice.
Variational data assimilation optimizes for an initial state of a dynamical system such that its evolution fits observational data. The physical model can subsequently be evolved into the future to make predictions. This principle is a…
Before we attempt to learn a function between two (sets of) observables of a physical process, we must first decide what the inputs and what the outputs of the desired function are going to be. Here we demonstrate two distinct, data-driven…
Scientific machine learning has become an increasingly important tool in materials science and engineering. It is particularly well suited to tackle material problems involving many variables or to allow rapid construction of surrogates of…
Collected data, which is used for analysis or prediction tasks, often have a hierarchical structure, for example, data from various people performing the same task. Modeling the data's structure can improve the reliability of the derived…
In this article, it is described how to use statistical data analysis to obtain models directly from data. The focus is put on finding nonlinearities within a generalized additive model. These models are found by the means of backfitting…
Conventional statistics begins with a model, and assigns a likelihood of obtaining any particular set of data. The opposite approach, beginning with the data and assigning a likelihood to any particular model, is explored here for the case…
In this study, two classes of methods including statistical and variational data assimilation algorithms will be described. In statistical methods, the model state is updated sequentially based on the previous estimate. Variational methods,…