Related papers: Information theory and discriminative sampling for…

The Fisher information matrix (FIM) plays an essential role in statistics and machine learning as a Riemannian metric tensor or a component of the Hessian matrix of loss functions. Focusing on the FIM and its variants in deep neural…

Discrete events alter how parameter influence propagates in hybrid systems. Prevailing Fisher information formulations assume that sensitivities evolve smoothly according to continuous-time variational equations and therefore neglect the…

The sparse identification of nonlinear dynamics (SINDy) has been established as an effective method to learn interpretable models of dynamical systems from data. However, for high-dimensional slow-fast dynamical systems, the regression…

Sparse model identification enables the discovery of nonlinear dynamical systems purely from data; however, this approach is sensitive to noise, especially in the low-data limit. In this work, we leverage the statistical approach of…

We present a Fisher-information (FI) framework for photon-efficient fluorescence lifetime imaging microscopy (FLIM) that treats temporal sampling as a controllable design variable under a fixed photon (dose) budget. Starting from a Poisson…

We describe how to analyze the wide class of non stationary processes with stationary centered increments using Shannon information theory. To do so, we use a practical viewpoint and define ersatz quantities from time-averaged probability…

Pattern-forming systems can exhibit a diverse array of complex behaviors as external parameters are varied, enabling a variety of useful functions in biological and engineered systems. First-principles derivations of the underlying…

The information-theoretic representation of quantum systems, which complements the familiar energy description of the density-functional and wave-function-based theories, is here discussed. According to it, the internal disorder of the…

Power grid parameter estimation involves the estimation of unknown parameters, such as inertia and damping coefficients, using observed dynamics. In this work, we present a comparison of data-driven algorithms for the power grid parameter…

The instantaneous relative entropy (IRE) and the corresponding instanta- neous Fisher information matrix (IFIM) for transient stochastic processes are pre- sented in this paper. These novel tools for sensitivity analysis of stochastic…

We determine the amount of information contained in a time series of price returns at a given time scale, by using a widespread tool of the information theory, namely the Shannon entropy, applied to a symbolic representation of this time…

The sparse identification of nonlinear dynamics (SINDy) approach can discover the governing equations of dynamical systems based on measurement data, where the dynamical model is identified as the sparse linear combination of the given…

We present a novel and simple method to numerically calculate Fisher Information Matrices for stochastic chemical kinetics models. The linear noise approximation is used to derive model equations and a likelihood function which leads to an…

The combination of machine learning (ML) and sparsity-promoting techniques is enabling direct extraction of governing equations from data, revolutionizing computational modeling in diverse fields of science and engineering. The discovered…

While frameworks based on physical grounds (like the Drift-Diffusion Model) have been exhaustively used in psychology and neuroscience to describe perceptual decision-making in humans, analogous approaches for more complex situations like…

Maximum entropy estimation is of broad interest for inferring properties of systems across many different disciplines. In this work, we significantly extend a technique we previously introduced for estimating the maximum entropy of a set of…

In this era of Big Data, proficient use of data mining is the key to capture useful information from any dataset. As numerous data mining techniques make use of information theory concepts, in this paper, we discuss how Fisher information…

An information theory description of finite systems explicitly evolving in time is presented. We impose a MaxEnt variational principle on the Shannon entropy at a given time while the constraints are set at a former time. The resulting…

Distributed systems, such as biological and artificial neural networks, process information via complex interactions engaging multiple subsystems, resulting in high-order patterns with distinct properties across scales. Investigating how…

Shannon's information entropy measures of the uncertainty of an event's outcome. If learning about a system reflects a decrease in uncertainty, then a plausible intuition is that learning should be accompanied by a decrease in the entropy…