Related papers: Quantifying synergistic information using intermed…
In stochastic simulation, input uncertainty refers to the output variability arising from the statistical noise in specifying the input models. This uncertainty can be measured by a variance contribution in the output, which, in the…
While mutual information effectively quantifies dependence between two variables, it does not by itself reveal the complex, fine-grained interactions among variables, i.e., how multiple sources contribute redundantly, uniquely, or…
The Shannon entropy, one of the cornerstones of information theory, is widely used in physics, particularly in statistical mechanics. Yet its characterization and connection to physics remain vague, leaving ample room for misconceptions and…
Data informativity provides a theoretical foundation for determining whether collected data are sufficiently informative to achieve specific control objectives in data-driven control frameworks. In this study, we investigate the data…
The entrainment between weakly-coupled nonlinear oscillators, as well as between complex signals such as those representing physiological activity, is frequently assessed in terms of whether a stable relationship is detectable between the…
Inferring patterns of synchronous brain activity from a heterogeneous sample of electroencephalograms (EEG) is scientifically and methodologically challenging. While it is intuitively and statistically appealing to rely on readings from…
The characterisation of information processing is an important task in complex systems science. Information dynamics is a quantitative methodology for modelling the intrinsic information processing conducted by a process represented as a…
Applying a machine learning model for decision-making in the real world requires to distinguish what the model knows from what it does not. A critical factor in assessing the knowledge of a model is to quantify its predictive uncertainty.…
We study semi-supervised stochastic optimization when labeled data is scarce but predictions from pre-trained models are available. PPI and SVRG both reduce variance through control variates -- PPI uses predictions, SVRG uses reference…
We introduce a novel notion of invariance feedback entropy to quantify the state information that is required by any controller that enforces a given subset of the state space to be invariant. We establish a number of elementary properties,…
The Error-in-Variables model of system identification/control involves nontrivial input and measurement corruption of observed data, resulting in generically nonconvex optimization problems. This paper performs full-state-feedback…
Permutation entropy quantifies the diversity of possible orderings of the values a random or deterministic system can take, as Shannon entropy quantifies the diversity of values. We show that the metric and permutation entropy…
Mutual information is a widely-used information theoretic measure to quantify the amount of association between variables. It is used extensively in many applications such as image registration, diagnosis of failures in electrical machines,…
In this paper we develop new data informativity based controller synthesis methods that extend existing frameworks in two relevant directions: a more general noise characterization in terms of cross-covariance bounds and informativity…
This paper proposes that the mathematical relationship between an entropy distribution and its limit offers some new insight into system performance. This relationship is used to quantify variation among the entities of a system, where…
Identifying the origin of nonequilibrium characteristics in a generic interacting system having multiple degrees of freedom is a challenging task. In this context, information theoretic measures such as mutual information and related…
The tension between qualitative theorizing and quantitative methods is pervasive in the social sciences, and poses a constant challenge to empirical research. But in science studies as an interdisciplinary specialty, there are additional…
One central goal of design of observational studies is to embed non-experimental data into an approximate randomized controlled trial using statistical matching. Despite empirical researchers' best intention and effort to create…
Estimating mutual correlations between random variables or data streams is essential for intelligent behavior and decision-making. As a fundamental quantity for measuring statistical relationships, mutual information has been extensively…
In his 1948 seminal paper A Mathematical Theory of Communication that birthed information theory, Claude Shannon introduced mutual information (MI), which he called "rate of transmission", as a way to quantify information gain (IG) and…