Related papers: Using Entropy-Based Methods to Study General Const…

In most data-scientific approaches, the principle of Maximum Entropy (MaxEnt) is used to a posteriori justify some parametric model which has been already chosen based on experience, prior knowledge or computational simplicity. In a…

Importance sampling of target probability distributions belonging to a given convex class is considered. Motivated by previous results, the cost of importance sampling is quantified using the relative entropy of the target with respect to…

We address an optimization problem where the cost function is the expectation of a random mapping. To tackle the problem two approaches based on the approximation of the objective function by consensus-based particle optimization methods on…

From a machine learning point of view, identifying a subset of relevant features from a real data set can be useful to improve the results achieved by classification methods and to reduce their time and space complexity. To achieve this…

We give some results relating asymptotic characterisations of maximum entropy probability measures to characterisations of Bayes optimal classifiers. Our main theorems show that maximum entropy is a universally Bayes optimal decision rule…

The major problem in information theoretic analysis of neural responses and other biological data is the reliable estimation of entropy--like quantities from small samples. We apply a recently introduced Bayesian entropy estimator to…

The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible while constrained to match empirically estimated feature expectations. However, in many real-world…

The maximum entropy principle advocates to evaluate events' probabilities using a distribution that maximizes entropy among those that satisfy certain expectations' constraints. Such principle can be generalized for arbitrary decision…

When looking for a solution, deterministic methods have the enormous advantage that they do find global optima. Unfortunately, they are very CPU-intensive, and are useless on untractable NP-hard problems that would require thousands of…

We consider dynamical systems evolving near an equilibrium statistical state where the interest is in modelling long term behavior that is consistent with thermodynamic constraints. We adjust the distribution using an entropy-optimizing…

In this paper, we propose an original approach to stochastic control problems. We consider a weak formulation that is written as an optimization (minimization) problem on the space of probability measures. We then introduce a penalized…

It is known that statistical model selection as well as identification of dynamical equations from available data are both very challenging tasks. Physical systems behave according to their underlying dynamical equations which, in turn, can…

Entanglement entropy is one of the most prominent measures in quantum physics. We show that it has an interesting ergotropic interpretation in terms of unitarily extracted work. It determines how much energy one can extract from a source of…

This is a review on entropy in various fields of mathematics and science. Its scope is to convey a unified vision of the classical as well as some newer entropy notions to a broad audience with an intermediate background in dynamical…

We exploit the idea to use the maximal-entropy method, successfully tested in information theory and statistical thermodynamics, to determine approximating function's coefficients and squared errors' weights simultaneously as output of one…

The optimization of composition and processing to obtain materials that exhibit desirable characteristics has historically relied on a combination of scientist intuition, trial and error, and luck. We propose a methodology that can…

The traditional way of tackling discrete optimization problems is by using local search on suitably defined cost or fitness landscapes. Such approaches are however limited by the slowing down that occurs when the local minima that are a…

Bayesian optimization works effectively optimizing parameters in black-box problems. However, this method did not work for high-dimensional parameters in limited trials. Parameters can be efficiently explored by nonlinearly embedding them…

In many engineered systems, optimization is used for decision making at time-scales ranging from real-time operation to long-term planning. This process often involves solving similar optimization problems over and over again with slightly…

We consider the optimization of an uncertain objective over continuous and multi-dimensional decision spaces in problems in which we are only provided with observational data. We propose a novel algorithmic framework that is tractable,…