Related papers: High-temperature Expansions and Message Passing Al…
The maximum entropy method is shown to be a special limit of the stochastic analytic continuation method introduced by Sandvik [Phys. Rev. B 57, 10287 (1998)]. We employ a mapping between the analytic continuation problem and a system of…
We sketch a new framework for the analysis of disordered systems, in particular mean field spin glasses, which is variational in nature and within the formalism of classical thermodynamics. For concreteness, only the Sherrington-Kirkpatrick…
The paper proposes a new message passing algorithm for cycle-free factor graphs. The proposed "entropy message passing" (EMP) algorithm may be viewed as sum-product message passing over the entropy semiring, which has previously appeared in…
Approximating marginals of a graphical model is one of the fundamental problems in the theory of networks. In a recent paper a method was shown to construct a variational free energy such that the linear response estimates, and maximum…
Embeddings are a basic initial feature extraction step in many machine learning models, particularly in natural language processing. An embedding attempts to map data tokens to a low-dimensional space where similar tokens are mapped to…
Consider the problem of estimating a low-rank matrix when its entries are perturbed by Gaussian noise. If the empirical distribution of the entries of the spikes is known, optimal estimators that exploit this knowledge can substantially…
We develop a generalized gradient expansion of the inhomogeneous dynamical mean-field theory method for determining properties of ultracold atoms in a trap. This approach goes beyond the well-known local density approximation and at higher…
The problem of assigning probability distributions which objectively reflect the prior information available about experiments is one of the major stumbling blocks in the use of Bayesian methods of data analysis. In this paper the method of…
This paper presents a new approach, called perturb-max, for high-dimensional statistical inference that is based on applying random perturbations followed by optimization. This framework injects randomness to maximum a-posteriori (MAP)…
Approximate message passing (AMP) is an algorithmic framework for solving linear inverse problems from noisy measurements, with exciting applications such as reconstructing images, audio, hyper spectral images, and various other signals,…
Approximate message passing (AMP) emerges as an effective iterative paradigm for solving high-dimensional statistical problems. However, prior AMP theory -- which focused mostly on high-dimensional asymptotics -- fell short of predicting…
We propose a high-dimensional extension of the heteroscedasticity test proposed in Newey and Powell (1987). Our test is based on expectile regression in the proportional asymptotic regime where n/p \to \delta \in (0,1]. The asymptotic…
Generative models provide a powerful framework for probabilistic reasoning. However, in many domains their use has been hampered by the practical difficulties of inference. This is particularly the case in computer vision, where models of…
Expectation Propagation (Minka, 2001) is a widely successful algorithm for variational inference. EP is an iterative algorithm used to approximate complicated distributions, typically to find a Gaussian approximation of posterior…
The canonical problem of solving a system of linear equations arises in numerous contexts in information theory, communication theory, and related fields. In this contribution, we develop a solution based upon Gaussian belief propagation…
We develop a resummed high-temperature expansion for lattice spin systems with long range interactions, in models where the free energy is not, in general, analytic. We establish uniqueness of the Gibbs state and exponential decay of the…
We consider classical $O(N)$ vector models in dimension three and higher and investigate the nature of the low-temperature expansions for their multipoint spin correlations. We prove that such expansions define asymptotic series, and derive…
This monograph deals with adaptive supervised classification, using tools borrowed from statistical mechanics and information theory, stemming from the PACBayesian approach pioneered by David McAllester and applied to a conception of…
The applicability conditions of a recently reported Central Limit Theorem-based approximation method in statistical physics are investigated and rigorously determined. The failure of this method at low and intermediate temperature is proved…
Analysis of extremal behavior of stochastic processes is a key ingredient in a wide variety of applications, including probability, statistical physics, theoretical computer science, and learning theory. In this paper, we consider centered…