Related papers: Macroscopic time evolution and MaxEnt inference fo…
Prediction is a central task of statistics and machine learning, yet many inferential settings provide only partial information, typically in the form of moment constraints or estimating equations. We develop a finite, fully Bayesian…
To make informed decisions in natural environments that change over time, humans must update their beliefs as new observations are gathered. Studies exploring human inference as a dynamical process that unfolds in time have focused on…
In the analysis of any type of system, granting maximum information extraction from its data is non-trivial. Confidence in successful information extraction typically builds on prior knowledge of the studied system or on the user's…
Time-arrow $s=+/-$, intrinsic to a concrete physical system, is associated with the direction of information loss $\Delta I$ displayed by the random evolution of the given system. When the information loss tends to zero the intrinsic…
We extend present Shannon's static statistical information theory to dynamic processes and establish a dynamic statistical information theory. We derive the nonlinear evolution equations of dynamic information density and dynamic…
We will discuss the maximum entropy production (MaxEP) principle based on Jaynes' information theoretical arguments, as was done by Dewar (2003, 2005). With the help of a simple mathematical model of a non-equilibrium system, we will show…
We consider the problem of tracking an unknown time varying parameter that characterizes the probabilistic evolution of a sequence of independent observations. To this aim, we propose a stochastic gradient descent-based recursive scheme in…
Life on earth is distinguished by long-lived correlations in time. The patterns of material organization that characterize living organisms today are contingent on events that occurred billions of years ago. This contingency is a necessary…
For a certain moment, the information volume represented in a probability space can be accurately measured by Shannon entropy. But in real life, the results of things usually change over time, and the prediction of the information volume…
Deep directed generative models have attracted much attention recently due to their expressive representation power and the ability of ancestral sampling. One major difficulty of learning directed models with many latent variables is the…
The well known maximum-entropy principle due to Jaynes, which states that given mean parameters, the maximum entropy distribution matching them is in an exponential family, has been very popular in machine learning due to its "Occam's…
We extend the notion of estimation entropy of autonomous dynamical systems proposed by Liberzon and Mitra [1] to nonlinear dynamical systems with uncertain inputs with bounded variation. We call this new notion the {$\epsilon$}-estimation…
We propose a method to derive the stationary size distributions of a system, and the degree distributions of networks, using maximisation of the Gibbs-Shannon entropy. We apply this to a preferential attachment-type algorithm for systems of…
We study the problem of synthesizing a policy that maximizes the entropy of a Markov decision process (MDP) subject to a temporal logic constraint. Such a policy minimizes the predictability of the paths it generates, or dually, maximizes…
Adaptive physical and biological systems continually process fluctuating information from their environments. When the environment is nonstationary, inference itself becomes a nonequilibrium process with thermodynamic cost. We analyse a…
Filyokov and Karpov [Inzhenerno-Fizicheskii Zhurnal 13, 624 (1967)] have proposed a theory of non-equilibrium steady states in direct analogy with the theory of equilibrium states : the principle is to maximize the Shannon entropy…
We introduce and analyze a model for the transport of particles or energy in extended lattice systems. The dynamics of the model acts on a discrete phase space at discrete times but has nonetheless some of the characteristic properties of…
In this work, we propose a hyperparameter optimization method named \emph{HyperTime} to find hyperparameters robust to potential temporal distribution shifts in the unseen test data. Our work is motivated by an important observation that it…
The problem of the insensitivity of the macroscopic behavior of any thermodynamical system to partitioning generates a bias between the reproducibility of its macroscopic behavior viewed as the simplest form of causality and its long-term…
Complex systems are sometimes subject to non Gaussian alpha stable Levy fluctuations. A new method is devised to estimate this uncertain parameter and other system parameters, using observations on either mean exit time or escape…