Related papers: Generalized maximum entropy estimation
Efficient approximation lies at the heart of large-scale machine learning problems. In this paper, we propose a novel, robust maximum entropy algorithm, which is capable of dealing with hundreds of moments and allows for computationally…
We describe a method to computationally estimate the probability density function of a univariate random variable by applying the maximum entropy principle with some local conditions given by Gaussian functions. The estimation errors and…
We propose a stochastic approximation method for approximating the efficient frontier of chance-constrained nonlinear programs. Our approach is based on a bi-objective viewpoint of chance-constrained programs that seeks solutions on the…
In the paper, we introduce the maximum entropy estimator based on 2-dimensional empirical distribution of the observation sequence of hidden Markov model , when the sample size is big: in that case computing the maximum likelihood estimator…
Moment-closure methods are popular tools to simplify the mathematical analysis of stochastic models defined on networks, in which high dimensional joint distributions are approximated (often by some heuristic argument) as functions of lower…
The classical problem of moments is addressed by the maximum entropy approach for one-dimensional discrete distributions. The numerical technique of adaptive support approximation is proposed to reconstruct the distributions in the region…
The maximum entropy principle advocates to evaluate events' probabilities using a distribution that maximizes entropy among those that satisfy certain expectations' constraints. Such principle can be generalized for arbitrary decision…
This paper presents a detailed theoretical analysis of the three stochastic approximation proximal gradient algorithms proposed in our companion paper [49] to set regularization parameters by marginal maximum likelihood estimation. We prove…
This paper shows how to evolve numerically the maximum entropy probability distributions for a given set of constraints, which is a variational calculus problem. An evolutionary algorithm can obtain approximations to some well-known…
Parameter estimation in Markov random fields (MRFs) is a difficult task, in which inference over the network is run in the inner loop of a gradient descent procedure. Replacing exact inference with approximate methods such as loopy belief…
This paper tackles the challenge of parameter calibration in stochastic models, particularly in scenarios where the likelihood function is unavailable in an analytical form. We introduce a gradient-based simulated parameter estimation…
Maximum entropy method is a constructive criterion for setting up a probability distribution maximally non-committal to missing information on the basis of partial knowledge, usually stated as constrains on expectation values of some…
The maximum entropy principle is a powerful tool for solving underdetermined inverse problems. This paper considers the problem of discretizing a continuous distribution, which arises in various applied fields. We obtain the approximating…
We present a technique for entropy optimization to calculate a distribution from its moments. The technique is based upon maximizing a discretized form of the Shannon entropy functional by mapping the problem onto a dual space where an…
The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible constrained to match empirical data, for instance, feature expectations. We seek to generalize…
Optimisation problems in science and engineering typically involve finding the ground state (i.e. the minimum energy configuration) of a cost function with respect to many variables. If the variables are corrupted by noise then this…
We propose a method of approximating multivariate Gaussian probabilities using dynamic programming. We show that solving the optimization problem associated with a class of discrete-time finite horizon Markov decision processes with…
It is well known that open dynamical systems can admit an uncountable number of (absolutely continuous) conditionally invariant measures (ACCIMs) for each prescribed escape rate. We propose and illustrate a convex optimisation based…
We study entropy-regularized constrained Markov decision processes (CMDPs) under the soft-max parameterization, in which an agent aims to maximize the entropy-regularized value function while satisfying constraints on the expected total…
The method of maximum entropy is quite a powerful tool to solve the generalized moment problem, which consists of determining the probability density of a random variable X from the knowledge of the expected values of a few functions of the…