Related papers: Optimizing entropy relative to a channel or a suba…

We introduce an axiomatic approach to entropies and relative entropies that relies only on minimal information-theoretic axioms, namely monotonicity under mixing and data-processing as well as additivity for product distributions. We find…

Inspired by the ideas from the field of stochastic approximation, we propose a randomized algorithm to compute the capacity of a finite-state channel with a Markovian input. When the mutual information rate of the channel is concave with…

We study the convexity of the entropy functional along particular interpolating curves defined on the space of finitely supported probability measures on a graph.

A central issue of the science of complex systems is the quantitative characterization of complexity. In the present work we address this issue by resorting to information geometry. Actually we propose a constructive way to associate to a -…

The familiar second derivative test for convexity, combined with resolvent calculus, is shown to yield a useful tool for the study of convex matrix-valued functions. We demonstrate the applicability of this approach on a number of theorems…

The analyticity of the entropy and relative entropy rates of continuous-state hidden Markov models is studied here. Using the analytic continuation principle and the stability properties of the optimal filter, the analyticity of these rates…

Evaluating the channel capacity is one of many key problems in information theory. In this work we derive rather-mild sufficient conditions under which the capacity is finite and achievable. These conditions are derived for generic,…

Entropy is a measure of self-information which is used to quantify losses. Entropy was developed in thermodynamics, but is also used to compare probabilities based on their deviating information content. Corresponding model uncertainty is…

Optimization problems with convex quadratic cost and polyhedral constraints are ubiquitous in signal processing, automatic control and decision-making. We consider here an enlarged problem class that allows to encode logical conditions and…

We simplify some conjectures in quantum information theory; the additivity of minimal output entropy, the multiplicativity of maximal output p-norm and the superadditivity of convex closure of output entropy. We construct a unital channel…

In this note we provide a simple proof of some properties enjoyed by convex functions having the engulfing property. In particular, making use only of results peculiar to convex analysis, we prove that differentiability and strict convexity…

Tight lower and upper bounds on the ratio of relative entropies of two probability distributions with respect to a common third one are established, where the three distributions are collinear in the standard $(n-1)$-simplex. These bounds…

Convex optimization problems arise naturally in quantum information theory, often in terms of minimizing a convex function over a convex subset of the space of hermitian matrices. In most cases, finding exact solutions to these problems is…

In this contribution, models of wireless channels are derived from the maximum entropy principle, for several cases where only limited information about the propagation environment is available. First, analytical models are derived for the…

We prove a lemma which allows one to extend results about the additivity of the minimal output entropy from highly symmetric channels to a much larger class. A similar result holds for the maximal output $p$-norm. Examples are given showing…

Various properties of relative entropy have led to its widespread use in information theory. These properties suggest that relative entropy has a role to play in systems that attempt to perform inference in terms of probability…

In this letter we propose the use of physics techniques for entropy determination on constrained parameter optimization problems. The main feature of such techniques, the construction of an unbiased walk on energy space, suggests their use…

The thermodynamic definition of entropy can be extended to nonequilibrium systems based on its relation to information. To apply this definition in practice requires access to the physical system's microstates, which may be prohibitively…

We prove some new properties of fidelity (transition probability) and concurrence, the latter defined by straightforward extension of Wootters notation. Choose a conjugation and consider the dependence of fidelity or of concurrence on…

We prove that under certain mild assumptions, the entropy rate of a hidden Markov chain, observed when passing a finite-state stationary Markov chain through a discrete-time continuous-output channel, is jointly analytic as a function of…