Related papers: Integrated information and dimensionality in conti…
Low-dimensional representations of underdamped systems often provide insightful grasps and analytical tractability. Here, we build such representations via information projections, obtaining an optimal model that captures the most…
Abrupt transitions ("tipping") in nonlinear dynamical systems are often accompanied by changes in the geometry of the attracting set, but quantifying such changes from partial and noisy observations in high-dimensional systems remains…
In this essay, a general case of information systems contains quantum information systems is considered. By presenting an algorithmic method a new kind of information topology is defined and considered. Continuous maps between two…
Information theory provides tools to predict the performance of a learning algorithm on a given dataset. For instance, the accuracy of learning an unknown parameter can be upper bounded by reducing the learning task to hypothesis testing…
The dynamics of many-body systems can often be captured in terms of only a few relevant variables. Mathematical and numerical approaches exist to identify these variables by exploiting a separation of time scales between slow relevant and…
Distributed computation in artificial life and complex systems is often described in terms of component operations on information: information storage, transfer and modification. Information modification remains poorly described however,…
Bayesian integral functional measure of entropy-uncertainty (EF) on trajectories of Markov multi-dimensional diffusion process is cutting off by interactive impulses (controls). Each cutoff minimax of EF superimposes and entangles…
Semantic Information Theory (SIT) offers a new approach to evaluating the information architecture of complex systems. In this study we describe the steps required to {\it operationalize} SIT via its application to dynamical problems. Our…
A framework integrating information theory and network science is proposed, giving rise to a potentially new area. By incorporating and integrating concepts such as complexity, coding, topological projections and network dynamics, the…
Various kinds of data are routinely represented as discrete probability distributions. Examples include text documents summarized by histograms of word occurrences and images represented as histograms of oriented gradients. Viewing a…
This paper provides a perspective on applying the concepts of information thermodynamics, developed recently in non-equilibrium statistical physics, to problems in theoretical neuroscience. Historically, information and energy in…
Activity classification has observed great success recently. The performance on small dataset is almost saturated and people are moving towards larger datasets. What leads to the performance gain on the model and what the model has learnt?…
Complex systems and relational data are often abstracted as dynamical processes on networks. To understand, predict and control their behavior, a crucial step is to extract reduced descriptions of such networks. Inspired by notions from…
The problems of causality, modeling, and control for chaotic, high-dimensional dynamical systems are formulated in the language of information theory. The central quantity of interest is the Shannon entropy, which measures the amount of…
Information theory is a practical and theoretical framework developed for the study of communication over noisy channels. Its probabilistic basis and capacity to relate statistical structure to function make it ideally suited for studying…
In this paper, we propose a unified information theoretic framework for learning-motivated methods aimed at odometry estimation, a crucial component of many robotics and vision tasks such as navigation and virtual reality where relative…
The uniform information density (UID) hypothesis posits a preference among language users for utterances structured such that information is distributed uniformly across a signal. While its implications on language production have been well…
Low rank tensor completion is a well studied problem and has applications in various fields. However, in many real world applications the data is dynamic, i.e., new data arrives at different time intervals. As a result, the tensors used to…
Appealing to several multivariate information measures---some familiar, some new here---we analyze the information embedded in discrete-valued stochastic time series. We dissect the uncertainty of a single observation to demonstrate how the…
Estimating the dimensionality of the latent representation needed for prediction -- the task-relevant dimension -- is a difficult, largely unsolved problem with broad scientific applications. We cast it as an Information Bottleneck…