Related papers: Extraction of physical laws from joint experimenta…
By considering the "kinetic-energy" term of the minimum principle for the Schr\"{o}dinger equation as a measure of information, that minimum principle is viewed as a statistical estimation procedure, analogous to the manner in which Jaynes…
This is an attempt to address diffusion phenomena from the point of view of information theory. We imagine a regular hamiltonian system under the random perturbation of thermal (molecular) noise and chaotic instability. The irregularity of…
We demonstrate that the processes underlying on-line auction price bids and many other longitudinal data can be represented by an empirical first order stochastic ordinary differential equation with time-varying coefficients and a smooth…
Learning disentangled representations of textual data is essential for many natural language tasks such as fair classification, style transfer and sentence generation, among others. The existent dominant approaches in the context of text…
The forced rupture of single chemical bonds under external load is addressed. A general framework is put forward to optimally utilize the experimentally observed rupture force data for estimating the parameters of a theoretical model. As an…
Extropy and its properties are explored to quantify the uncertainty. In this paper, we obtain alternative expressions for cumulative residual extropy and negative cumulative extropy. We obtain simple estimators of cumulative (residual)…
Mixture distributions are extensively used as a modeling tool in diverse areas from machine learning to communications engineering to physics, and obtaining bounds on the entropy of probability distributions is of fundamental importance in…
Mutual information is fundamentally important for measuring statistical dependence between variables and for quantifying information transfer by signaling and communication mechanisms. It can, however, be challenging to evaluate for…
We define {\em predictive information} $I_{\rm pred} (T)$ as the mutual information between the past and the future of a time series. Three qualitatively different behaviors are found in the limit of large observation times $T$: $I_{\rm…
We propose and analyze a process that extracts useful work from a single active particle maintained at constant temperature in a harmonic potential by measuring the relative sign of the self-propulsion and the confining force and then…
Fields like public health, public policy, and social science often want to quantify the degree of dependence between variables whose relationships take on unknown functional forms. Typically, in fact, researchers in these fields are…
We have discovered analytical expressions for the probability density function (PDF) of photons that are multiply scattered in relativistic flows, under the assumption of isotropic and inelastic scattering. These expressions characterize…
Estimating the entropy of a discrete random variable is a fundamental problem in information theory and related fields. This problem has many applications in various domains, including machine learning, statistics and data compression. Over…
The aim of this paper is to investigate various information-theoretic measures, including entropy, mutual information, and some systematic measures that based on mutual information, for a class of structured spiking neuronal network. In…
We propose a test of independence of two multivariate random vectors, given a sample from the underlying population. Our approach, which we call MINT, is based on the estimation of mutual information, whose decomposition into joint and…
A one-to-one correspondence is drawn between law invariant risk measures and divergences, which we define as functionals of pairs of probability measures on arbitrary standard Borel spaces satisfying a few natural properties. Divergences…
Dependence among marginally constrained observations can break a finite-sample barrier. To formalize this phenomenon, we introduce the \emph{minimum list entropy coupling} $H(P\|Q_1,\dots,Q_m)$, the minimum conditional entropy…
The joint probability density function (PDF) of turbulent velocity and concentration of a passive scalar in an urban street canyon is computed using a newly developed particle-in-cell Monte Carlo method. Compared to moment closures, the PDF…
Mutual information is a widely-used information theoretic measure to quantify the amount of association between variables. It is used extensively in many applications such as image registration, diagnosis of failures in electrical machines,…
There are interesting extensions of the problem of determining a joint probability with known marginals. On the one hand, one may impose size constraints on the joint probabilities. On the other, one may impose additional constraints like…