Related papers: Integrating hidden information which is observed a…
We propose two different approaches for introducing the information temperature of the binary N-th order Markov chains. The first approach is based on comparing the Markov sequences with the equilibrium Ising chains at given temperatures.…
Information-theoretic (IT) measures are ubiquitous in artificial intelligence: entropy drives decision-tree splits and uncertainty quantification, cross-entropy is the default classification loss, mutual information underpins representation…
Information entropic measures such as Fisher information, Shannon entropy, Onicescu energy and Onicescu Shannon entropy of a symmetric double-well potential are calculated in both position and momentum space. Eigenvalues and eigenvectors of…
Accumulating evidence indicates that the capacity to integrate information in the brain is a prerequisite for consciousness. Integrated Information Theory (IIT) of consciousness provides a mathematical approach to quantifying the…
In this Thesis, several results in quantum information theory are collected, most of which use entropy as the main mathematical tool. *While a direct generalization of the Shannon entropy to density matrices, the von Neumann entropy behaves…
Although the notion of entropy lies at the core of statistical mechanics, it is not often used in statistical mechanical models to characterize phase transitions, a role more usually played by quantities such as various order parameters,…
Mutual information and information entropies in momentum space are proposed as measures of the non-local aspects of information. Singlet and triplet state members of the helium isoelectronic series are employed to examine Coulomb and Fermi…
Inference and learning are commonly cast in terms of optimisation, yet the fundamental constraints governing uncertainty reduction remain unclear. This work presents a first-principles framework inherent to Bayesian updating, termed…
We explore the dynamics of information systems. We show that the driving force for information dynamics is determined by both the information landscape and information flux which determines the equilibrium time reversible and the…
During a spontaneous change, a macroscopic physical system will evolve towards a macro-state with more realizations. This observation is at the basis of the Statistical Mechanical version of the Second Law of Thermodynamics, and it provides…
Mutual Information (MI) is a crucial measure for capturing dependencies between variables, but exact computation is challenging in high dimensions with intractable likelihoods, impacting accuracy and robustness. One idea is to use an…
Opacity, or non-interference, is a property ensuring that an external observer cannot infer confidential information (the "secret") from system observations. We introduce an information-theoretic measure of opacity, which quantifies…
Information theory provides tools to predict the performance of a learning algorithm on a given dataset. For instance, the accuracy of learning an unknown parameter can be upper bounded by reducing the learning task to hypothesis testing…
A novel information entropy of turbulence systems with multiple field quantities is formulated. Inspired by quantum mechanics, the von Neumann entropy (vNE) and the entanglement entropy(EE) are derived from a density matrix for the…
Integrated Information Theory (IIT) is a prominent theory of consciousness that has at its centre measures that quantify the extent to which a system generates more information than the sum of its parts. While several candidate measures of…
The apparent dichotomy between information-processing and dynamical approaches to complexity science forces researchers to choose between two diverging sets of tools and explanations, creating conflict and often hindering scientific…
The constituents of a complex system exchange information to function properly. Their signalling dynamics often leads to the appearance of emergent phenomena, such as phase transitions and collective behaviors. While information exchange…
Obtaining meaningful quantitative descriptions of the statistical dependence within multivariate systems is a difficult open problem. Recently, the Partial Information Decomposition (PID) was proposed to decompose mutual information (MI)…
Fluctuations in biochemical networks, e.g., in a living cell, have a complex origin that precludes a description of such systems in terms of bipartite or multipartite processes, as is usually done in the framework of stochastic and/or…
The characterisation of information processing is an important task in complex systems science. Information dynamics is a quantitative methodology for modelling the intrinsic information processing conducted by a process represented as a…