Related papers: Analyzing Training Using Phase Transitions in Entr…
A one-dimensional model on a line of the length L is investigated, which involves particle diffusion as well as single particle annihilation. There are also creation and annihilation at the boundaries. The static and dynamical behaviors of…
This paper explores the relationship between the condition number of a neural network's weight tensor and the extent of information encoded by the associated processing unit, viewed through the lens of information theory. It argues that a…
We translate the problem of calculating the entropy of a set of binary configurations/signals into a sequence of supervised classification tasks. Subsequently, one can use virtually any machine learning classification algorithm for…
Entropies are fundamental measures of uncertainty with central importance in information theory and statistics and applications across all the quantitative sciences. Under a natural set of operational axioms, the most general form of…
This paper looks at the early theory of phase transitions. It considers a group of related concepts derived from condensed matter and statistical physics. The key technical ideas here go under the names of "singularity", "order parameter",…
In this work, conditional entropy is used to quantify the information loss induced by passing a continuous random variable through a memoryless nonlinear input-output system. We derive an expression for the information loss depending on the…
If a given behavior of a multi-agent system restricts the phase variable to a invariant manifold, then we define a phase transition as change of physical characteristics such as speed, coordination, and structure. We define such a phase…
When first principle models cannot be derived due to the complexity of the real system, data-driven methods allow us to build models from system observations. As these models are employed in learning-based control, the quality of the data…
We investigate the macroscopic effects of the ingredients that drive the origin of species through sympatric speciation. In our model, sympatric speciation is obtained as we tune up the strength of competition between individuals with…
An approach for the description of stochastic systems is derived. Some of the variables in the system are studied forward in time, others backward in time. The approach is based on a perturbation expansion in the strength of the coupling…
The major problem in information theoretic analysis of neural responses and other biological data is the reliable estimation of entropy--like quantities from small samples. We apply a recently introduced Bayesian entropy estimator to…
Simulating transition dynamics between metastable states is a fundamental challenge in dynamical systems and stochastic processes with wide real-world applications in understanding protein folding, chemical reactions and neural activities.…
By combining different ideas, a general and efficient protocol to deal with discontinuous phase transitions at low temperatures is proposed. For small $T$'s, it is possible to derive a generic analytic expression for appropriate order…
Transfer entropy provides a general tool for analyzing the magnitudes and directions---but not the \emph{kinds}---of information transfer in a system. We extend transfer entropy in two complementary ways. First, we distinguish…
Phase transitions generically occur in random matrix models as the parameters in the joint probability distribution of the random variables are varied. They affect all main features of the theory and the interpretation of statistical models…
We study phase transformations in finite nuclei as a function of interaction parameters. The signature of a transition is given by invariant correlational entropy that reflects the sensitivity of an individual many-body state to changes of…
Maximum entropy estimation is of broad interest for inferring properties of systems across many different disciplines. In this work, we significantly extend a technique we previously introduced for estimating the maximum entropy of a set of…
This paper explores the connection between dynamical system properties and statistical physics of ensembles of such systems. Simple models are used to give novel phase transitions; particularly for finite N particle systems with many…
The gap in statistics between multi-variate and time-series analysis can be bridged by using entropy statistics and recent developments in multi-dimensional scaling. For explaining the evolution of the sciences as non-linear dynamics, the…
Neural networks have dramatically increased our capacity to learn from large, high-dimensional datasets across innumerable disciplines. However, their decisions are not easily interpretable, their computational costs are high, and building…