相关论文: Entropy Message Passing

Using a maximum entropy principle to assign a statistical weight to any graph, we introduce a model of random graphs with arbitrary degree distribution in the framework of standard statistical mechanics. We compute the free energy and the…

We propose a formal expansion of the transfer entropy to put in evidence irreducible sets of variables which provide information for the future state of each assigned target. Multiplets characterized by a large contribution to the expansion…

Envelope method was recently proposed as a method to reduce the dimension of responses in multivariate regressions. However, when there exists missing data, the envelope method using the complete case observations may lead to biased and…

The Principle of Maximum Entropy is a rigorous technique for estimating an unknown distribution given partial information while simultaneously minimizing bias. However, an important requirement for applying the principle is that the…

An efficient approach to the calculation of the $\epsilon$-entropy is proposed. The method is based on the idea of looking at the information content of a string of data, by analyzing the signal only at the instants when the fluctuations…

The estimation of a random vector with independent components passed through a linear transform followed by a componentwise (possibly nonlinear) output map arises in a range of applications. Approximate message passing (AMP) methods, based…

Message-passing algorithms have emerged as powerful techniques for approximate inference in graphical models. When these algorithms converge, they can be shown to find local (or sometimes even global) optima of variational formulations to…

The maximum entropy principle (MEP) is a method for obtaining the most likely distribution functions of observables from statistical systems, by maximizing entropy under constraints. The MEP has found hundreds of applications in ergodic and…

Characterizing the entropy of a system is a crucial, and often computationally costly, step in understanding its thermodynamics. It plays a key role in the study of phase transitions, pattern formation, protein folding and more. Current…

The Expectation-Maximization (EM) algorithm is one of the most popular methods used to solve the problem of parametric distribution-based clustering in unsupervised learning. In this paper, we propose to analyze a generalized EM (GEM)…

Hidden Markov Processes (HMP) is one of the basic tools of the modern probabilistic modeling. The characterization of their entropy remains however an open problem. Here the entropy of HMP is calculated via the cycle expansion of the…

R\'esum\'e: Le principal objet de cette communication est de faire une r\'etro perspective succincte de l'utilisation de l'entropie et du principe du maximum d'entropie dans le domaine du traitement du signal. Apr\`es un bref rappel de…

Active systems comprise a class of nonequilibrium dynamics in which individual components autonomously dissipate energy. Efforts towards understanding the role played by activity have centered on computation of the entropy production rate…

The Random Permutation Set (RPS) is a new type of set proposed recently, which can be regarded as the generalization of evidence theory. To measure the uncertainty of RPS, the entropy of RPS and its corresponding maximum entropy have been…

The classical EMD algorithm has been used extensively in the literature to decompose signals that contain nonlinear waves. However when a signal contain two or more frequencies that are close to one another the decomposition might fail. In…

We propose Embedding Propagation (EP), an unsupervised learning framework for graph-structured data. EP learns vector representations of graphs by passing two types of messages between neighboring nodes. Forward messages consist of label…

A new approach for signal parametrization, which consists of a specific regression model incorporating a discrete hidden logistic process, is proposed. The model parameters are estimated by the maximum likelihood method performed by a…

We introduce a graph-signal generalisation of Sample Entropy, denoted SampEn$_{G}$, to quantify irregularity of graph signals on a continuous state space, complementing existing methods on symbolic dynamics. Our approach replaces the…

We introduce and analyze the notion of mutual entropy-production (MEP) in autonomous systems. Evaluating MEP rates is in general a difficult task due to non-Markovian effects. For bipartite systems, we provide closed expressions in various…

In Monte Carlo simulation, lattice field theory with a $\theta$ term suffers from the sign problem. This problem can be circumvented by Fourier-transforming the topological charge distribution $P(Q)$. Although this strategy works well for…