Related papers: The Entropy of a Binary Hidden Markov Process
The problem of sampling according to the probability distribution minimizing a given free energy, using interacting particles unadjusted kinetic Langevin Monte Carlo, is addressed. In this setting, three sources of error arise, related to…
Living systems operate far from thermal equilibrium by converting the chemical potential of ATP into mechanical work to achieve growth, replication or locomotion. Given time series observations of intra-, inter- or multicellular processes,…
Microscopic formula to describe the entropy of biomolecular solutions are derived based on the Gibbs formula of entropy, and the generalized Langevin theory combined with the RISM/3D-RISM theory. Two formula are derived: one is concerned…
We consider the problem of the construction of the estimator-process of the unknown finite-dimensional parameter in the case of the observations of nonlinear autoregressive process. The estimation is done in two or three steps. First we…
The standard approach to analyzing the asymptotic complexity of probabilistic programs is based on studying the asymptotic growth of certain expected values (such as the expected termination time) for increasing input size. We argue that…
The encoder and decoder for lossy data compression of binary memoryless sources are developed on the basis of a specific-type nonmonotonic perceptron. Statistical mechanical analysis indicates that the potential ability of the…
We study the so-called two-time-scale stochastic approximation, a simulation-based approach for finding the roots of two coupled nonlinear operators. Our focus is to characterize its finite-time performance in a Markov setting, which often…
The problem of finding the expected value of a statistic of a locally stable point process in a bounded region is addressed. We propose an adaptive importance sampling for solving the problem. In our proposal, we restrict the importance…
This paper considers the speed of convergence (mixing) of a finite Markov kernel $P$ with respect to the Kullback-Leibler divergence (entropy). Given a Markov kernel one defines either a discrete-time Markov chain (with the $n$-step…
We introduce a systematic framework to calculate the bipartite entanglement entropy of a compact spatial subsystem in a one-dimensional quantum gas which can be mapped into a noninteracting fermion system. We show that when working with a…
A definition of entropy via the Kolmogorov algorithmic complexity is discussed. As examples, we show how the meanfield theory for the Ising model, and the entropy of a perfect gas can be recovered. The connection with computations are…
We consider corrections to the entropy of a black hole from an $O(N)$ invariant linear $\s$-model. We obtain the entropy from a $1/N$ expansion of the partition function on a cone. The entropy arises from diagrams which are analogous to…
Under continuity and recurrence assumptions, we prove that the iteration of successive partial symmetrizations that form a time-homogeneous Markov process, converges to a symmetrization. We cover several settings, including the…
Based on information theory, we present a method to determine an optimal Markov approximation for modelling and prediction from time series data. The method finds a balance between minimal modelling errors by taking as much as possible…
We study how large an $\ell^2$ ellipsoid is by introducing type-$\tau$ integrals that capture the average decay of its semi-axes. These integrals turn out to be closely related to standard complexity measures: we show that the metric…
In this paper we prove that the asymptotic rate of exponential loss of memory of a finite state hidden Markov model is bounded above by the difference of the first two Lyapunov exponents of a certain product of matrices. We also show that…
We generalize Gaspard's method for computing the \epsilon-entropy production rate in Hamiltonian systems to dissipative systems with attractors considered earlier by T\'el, Vollmer, and Breymann. This approach leads to a natural definition…
Behavior of the entropy numbers of classes of multivariate functions with mixed smoothness is studied here. This problem has a long history and some fundamental problems in the area are still open. The main goal of this paper is to develop…
Similarity-sensitive entropy measures the uncertainty of a probability law relative to a similarity kernel that encodes the distinguishability between states. We develop a measure-theoretic treatment covering both finite similarity matrices…
The paper describes an approach to measuring convergence of an algorithm to its result in terms of an entropy-like function of partitions of its inputs of a given length. The goal is to look at the algorithmic data processing from the…