Related papers: Updating Probabilities with Data and Moments
Many offline unsupervised change point detection algorithms rely on minimizing a penalized sum of segment-wise costs. We extend this framework by proposing to minimize a sum of discrepancies between segments. In particular, we propose to…
Formalising the confrontation of opinions (models) to observations (data) is the task of Inferential Statistics. Information Theory provides us with a basic functional, the relative entropy (or Kullback-Leibler divergence), an asymmetrical…
These lectures deal with the problem of inductive inference, that is, the problem of reasoning under conditions of incomplete information. Is there a general method for handling uncertainty? Or, at least, are there rules that could in…
What is information? Is it physical? We argue that in a Bayesian theory the notion of information must be defined in terms of its effects on the beliefs of rational agents. Information is whatever constrains rational beliefs and therefore…
Ill-posed inverse problems of the form y = X p where y is J-dimensional vector of a data, p is m-dimensional probability vector which cannot be measured directly and matrix X of observable variables is a known J,m matrix, J < m, are…
Herding is a deterministic algorithm used to generate data points that can be regarded as random samples satisfying input moment conditions. The algorithm is based on the complex behavior of a high-dimensional dynamical system and is…
The problem of overdispersion in multivariate count data is a challenging issue. Nowadays, it covers a central role mainly due to the relevance of modern technologies data, such as Next Generation Sequencing and textual data from the web or…
We establish the theoretical framework for implementing the maximumn entropy on the mean (MEM) method for linear inverse problems in the setting of approximate (data-driven) priors. We prove a.s. convergence for empirical means and further…
The diversity of a community that cannot be fully counted must be inferred. The two preeminent inference methods are the MaxEnt method, which uses information in the form of constraints and Bayes' rule which uses information in the form of…
Weighted Updating generalizes Bayesian updating, allowing for biased beliefs by weighting the likelihood function and prior distribution with positive real exponents. I provide a rigorous foundation for the model by showing that…
Belief updating in Bayes nets, a well known computationally hard problem, has recently been approximated by several deterministic algorithms, and by various randomized approximation algorithms. Deterministic algorithms usually provide…
The best practical techniques for exact solution of instances of the constrained maximum-entropy sampling problem, a discrete-optimization problem arising in the design of experiments, are via a branch-and-bound framework, working with a…
We consider the problem of guessing the realization of a random variable but under more general Tsallis' non-extensive entropic framework rather than the classical Maxwell-Boltzman-Gibbs-Shannon framework. We consider both the conditional…
If moments of singular measures are passed as inputs to the entropy maximization procedure, the optimization algorithm might not terminate. The framework developed in our previous paper demonstrated how input moments of measures, on a broad…
We outline an efficient method for the reconstruction of a probability density function from the knowledge of its infinite sequence of ordinary moments. The approximate density is obtained resorting to maximum entropy technique, under the…
The moment problem in probability theory asks for criteria for when there exists a unique measure with a given tuple of moments. We study a variant of this problem for random objects in a category, where a moment is given by the average…
A common problem in the optimization of structures is the handling of uncertainties in the parameters. If the parameters appear in the constraints, the uncertainties can lead to an infinite number of constraints. Usually the constraints…
We apply the Principle of Maximum Entropy to the study of a general class of deterministic fractal sets. The scaling laws peculiar to these objects are accounted for by means of a constraint concerning the average content of information in…
Learning conditional distributions $\pi^*(\cdot|x)$ is a central problem in machine learning, which is typically approached via supervised methods with paired data $(x,y) \sim \pi^*$. However, acquiring paired data samples is often…
Inverse classification, the process of making meaningful perturbations to a test point such that it is more likely to have a desired classification, has previously been addressed using data from a single static point in time. Such an…