Related papers: Entropy NAND: Early Functional Completeness in Ent…
Incremental information, as measured by the quantum entropy, is increasing when two ensembles are united. This result was proved by Lieb and Ruskai, and it is the foundation for the proof of strong subadditivity of quantum entropy. We…
We consider a section of a half-filled chain of free electrons and its entanglement with the rest of the system in the presence of one or two interface defects. We find a logarithmic behaviour of the entanglement entropy with constants…
Entropy is a measure of self-information which is used to quantify losses. Entropy was developed in thermodynamics, but is also used to compare probabilities based on their deviating information content. Corresponding model uncertainty is…
The principle of maximum entropy provides a useful method for inferring statistical mechanics models from observations in correlated systems, and is widely used in a variety of fields where accurate data are available. While the assumptions…
Current neural networks architectures are many times harder to train because of the increasing size and complexity of the used datasets. Our objective is to design more efficient training algorithms utilizing causal relationships inferred…
Entropy is the measure of uncertainty in any data and is adopted for maximisation of mutual information in many remote sensing operations. The availability of wide entropy variations motivated us for an investigation over the suitability…
Entropic measures of complexity are able to quantify the information encoded in complex network structures. Several entropic measures have been proposed in this respect. Here we study the relation between the Shannon entropy and the Von…
Consider a state of a system with several subsystems. The entropies of the reduced state on different subsystems obey certain inequalities, provided there is an equivalence relation, and a function measuring volumes or weights of…
We study entanglement entropy of excited states in two dimensional conformal field theories (CFTs). Especially we consider excited states obtained by acting primary operators on a vacuum. We show that under its time evolution, entanglement…
Inspired by the swarming or flocking of animal systems we study groups of agents moving in unbounded 2D space. Individual trajectories derive from a ``bottom-up'' principle: individuals reorient to maximise their future path entropy over…
The time variation of entropy, as an alternative to the variance, is proposed as a measure of the diffusion rate. It is shown that for linear and time-translationally invariant systems having a large-time limit for the density, at large…
By using relative entropy of coherence, we characterize the coherence gain induced by some quantum evolutions, including the cohering power of unitary operations and the decohering power of quantum operations. We find that the cohering…
Recently, there is a growing interest in applying Transfer Entropy (TE) in quantifying the effective connectivity between artificial neurons. In a feedforward network, the TE can be used to quantify the relationships between neuron output…
The probability distribution of the total entropy production in the non-equilibrium steady state follows a symmetry relation called the fluctuation theorem. When a certain part of the system is masked or hidden, it is difficult to infer the…
Link-prediction is an active research field within network theory, aiming at uncovering missing connections or predicting the emergence of future relationships from the observed network structure. This paper represents our contribution to…
We explore the connection between deep learning and information theory through the paradigm of diffusion models. A diffusion model converts noise into structured data by reinstating, imperfectly, information that is erased when data was…
In [1] it is shown that recurrent neural networks (RNNs) can learn - in a metric entropy optimal manner - discrete time, linear time-invariant (LTI) systems. This is effected by comparing the number of bits needed to encode the…
Identifying the status of individual network units is critical for understanding the mechanism of convolutional neural networks (CNNs). However, it is still challenging to reliably give a general indication of unit status, especially for…
The thermodynamic uncertainty relation originally proven for systems driven into a non-equilibrium steady state (NESS) allows one to infer the total entropy production rate by observing any current in the system. This kind of inference…
Entropy rate is a real valued functional on the space of discrete random sources which lacks a closed formula even for subclasses of sources which have intuitive parameterizations. A good way to overcome this problem is to examine its…