Related papers: Differential Properties of Information in Jump-dif…
We study diversity in one-shot communication over molecular timing channels. We consider a channel model where the transmitter simultaneously releases a large number of information particles, while the information is encoded in the time of…
In this paper, we review Fisher information matrices properties in weighted version and discuss inequalities/bounds on it by using reduced weight functions. In particular, an extended form of the Fisher information inequality previously…
We consider torsion in parameter manifolds that arises via conformal transformations of the Fisher information metric, and define it for information geometry of a wide class of physical systems. The torsion can be used to differentiate…
Estimating mutual information from observed samples is a basic primitive, useful in several machine learning tasks including correlation mining, information bottleneck clustering, learning a Chow-Liu tree, and conditional independence…
In diffusion-based molecular communications, messages can be conveyed via the variation in the concentration of molecules in the medium. In this paper, we intend to analyze the achievable capacity in transmission of information from one…
Prior information about the input state can be utilized to enhance the efficiency of quantum teleportation which we quantify using the first two moments of fidelity. The input knowledge is introduced by relaxing the uniformity assumption in…
Several recent works in communication systems have proposed to leverage the power of neural networks in the design of encoders and decoders. In this approach, these blocks can be tailored to maximize the transmission rate based on…
Denoising diffusion models have spurred significant gains in density modeling and image generation, precipitating an industrial revolution in text-guided AI art generation. We introduce a new mathematical foundation for diffusion models…
We study the generic scaling properties of the mutual information between two disjoint intervals, in a class of one-dimensional quantum critical systems described by the c=1 bosonic field theory. A numerical analysis of a spin-chain model…
In three-dimensional turbulence, information of turbulent fluctuations at large scales is propagated to small scales. Here, we investigate the relation between the information flow and turbulent fluctuations described by a shell model. We…
Multilayer (or deep) networks are powerful probabilistic models based on multiple stages of a linear transform followed by a non-linear (possibly random) function. In general, the linear transforms are defined by matrices and the non-linear…
The mutual information between two jointly distributed random variables $X$ and $Y$ is a functional of the joint distribution $P_{XY},$ which is sometimes difficult to handle or estimate. A coarser description of the statistical behavior of…
A novel information entropy of turbulence systems with multiple field quantities is formulated. Inspired by quantum mechanics, the von Neumann entropy (vNE) and the entanglement entropy(EE) are derived from a density matrix for the…
We present an information-theoretic framework for discrete diffusion models that yields principled estimators of log-likelihood using score-matching losses. Inspired by the I-MMSE identity for the Gaussian setup, we derive analogous results…
Shannon entropy and Fisher information functionals are known to quantify certain information-theoretic properties of continuous probability distributions of various origins. We carry out a systematic study of these functionals, while…
This paper studies the problem of secure communication over a K-transmitter multiple access channel in the presence of an external eavesdropper, subject to a joint secrecy constraint (i.e., information leakage rate from the collection of K…
The consistency across scales of a recently developed mathematical thermodynamic structure, between a continuous stochastic nonlinear dynamical system (diffusion process with Langevin or Fokker-Planck equations) and its emergent discrete,…
The dissipation of general convex entropies for continuous time Markov processes can be described in terms of backward martingales with respect to the tail filtration. The relative entropy is the expected value of a backward submartingale.…
Statistical divergences are important tools in data analysis, information theory, and statistical physics, and there exist well known inequalities on their bounds. However, in many circumstances involving temporal evolution, one needs…
Unveiling a fundamental link between information theory and estimation theory, the I-MMSE relation by Guo, Shamai and Verdu~\cite{gu05}, together with its numerous extensions, has great theoretical significance and various practical…