Related papers: A Nonstochastic Information Theory for Communicati…
Several types of biological networks have recently been shown to be accurately described by a maximum entropy model with pairwise interactions, also known as the Ising model. Here we present an approach for finding the optimal mappings…
This paper addresses the problem of distributed state estimation via multiple access channels (MACs). We consider a scenario where two encoders are simultaneously communicating their measurements through a noisy channel. Firstly, the…
We address the fundamental limits of learning unknown parameters of any stochastic process from time-series data, and discover exact closed-form expressions for how optimal inference scales with observation length. Given a parametrized…
In nervous system information is conveyed by sequence of action potentials (spikes-trains). As MacKay and McCulloch proposed, spike-trains can be represented as bits sequences coming from Information Sources. Previously, we studied…
We consider optimal signalling and control of discrete-time nonlinear partially observable stochastic systems in state space form. In the first part of the paper, we characterize the operational {\it control-coding capacity}, $C_{FB}$ in…
Prompted by the realisation that the statistical entropy of an ideal gas in the micro-canonical ensemble should not fluctuate or change over time, the meaning of the H-theorem is re-interpreted from the perspective of information theory in…
Parameter identification problems are formulated in a probabilistic language, where the randomness reflects the uncertainty about the knowledge of the true values. This setting allows conceptually easily to incorporate new information, e.g.…
Complex networks are mapped to a model of boxes and balls where the balls are distinguishable. It is shown that the scale-free size distribution of boxes maximizes the information associated with the boxes provided configurations including…
A path information is defined in connection with the different possible paths of chaotic system moving in its phase space between two cells. On the basis of the assumption that the paths are differentiated by their actions, we show that the…
Bounded agents are limited by intrinsic constraints on their ability to process information that is available in their sensors and memory and choose actions and memory updates. In this dissertation, we model these constraints as…
We consider the problem of least squares parameter estimation from single-trajectory data for discrete-time, unstable, closed-loop nonlinear stochastic systems, with linearly parameterised uncertainty. Assuming a region of the state space…
Statistical properties of coupled dynamic-stochastic systems are studied within a combination of the maximum information principle and the superstatistical approach. The conditions at which the Shannon entropy functional leads to a…
We study infinite-horizon stochastic optimal control problems with observable side information: a Markov chain that modulates an unknown context-conditional randomness distribution. Since this distribution is unknown, we propose a Bayesian…
In this paper, we consider the problem of optimizing the worst-case behavior of a partially observed system. All uncontrolled disturbances are modeled as finite-valued uncertain variables. Using the theory of cost distributions, we present…
We consider a finite-state memoryless channel with i.i.d. channel state and the input Markov process supported on a mixing finite-type constraint. We discuss the asymptotic behavior of entropy rate of the output hidden Markov chain and…
Given a universe of discourse X-a domain of possible outcomes-an experiment may consist of selecting one of its elements, subject to the operation of chance, or of observing the elements, subject to imprecision. A priori uncertainty about…
In stochastic modeling, the excess entropy -- the mutual information shared between a process's past and future -- represents the fundamental lower bound of the memory needed to simulate its dynamics. However, this bound cannot be saturated…
We consider the problem of identifying the most influential nodes for a spreading process on a network when prior knowledge about structure and dynamics of the system is incomplete or erroneous. Specifically, we perform a numerical analysis…
Understanding how network function constrains neural connectivity is a central challenge in neuroscience. An influential approach is to train neural networks with gradient descent on cognitive tasks and characterize the resulting…
We study a continuous time stochastic optimal control problem under partial observations that are available only at discrete time instants. This hybrid setting, with continuous dynamics and intermittent noisy measurements, arises in…