Related papers: Comparing Information-Theoretic Measures of Comple…
We introduce a class of information measures based on group entropies, allowing us to describe the information-theoretical properties of complex systems. These entropic measures are nonadditive, and are mathematically deduced from a series…
It is a challenge to predict the response of a large, complex system to a perturbation. Recent attempts to predict the behaviour of food webs have revealed that the effort needed to understand a system grows quickly with its complexity,…
Quantifying integrated information is a leading approach towards building a fundamental theory of consciousness. Integrated Information Theory (IIT) has gained attention in this regard due to its theoretically strong framework. However, it…
We analyze the generalization performance of a student in a model composed of nonlinear perceptrons: a true teacher, ensemble teachers, and the student. We calculate the generalization error of the student analytically or numerically using…
We investigate the relationship between complexity, information transfer and the emergence of collective behaviors, such as synchronization and nontrivial collective behavior, in a network of globally coupled chaotic maps as a simple model…
Observations on the past provide some hints about what will happen in the future, and this can be quantified using information theory. The ``predictive information'' defined in this way has connections to measures of complexity that have…
The problem of defining and studying complexity of a time series has interested people for years. In the context of dynamical systems, Grassberger has suggested that a slow approach of the entropy to its extensive asymptotic limit is a sign…
In contrast to entropy, which increases monotonically, the "complexity" or "interestingness" of closed systems seems intuitively to increase at first and then decrease as equilibrium is approached. For example, our universe lacked complex…
We introduce the Insertion Chain Complex, a higher-dimensional extension of insertion graphs, as a new framework for analyzing finite sets of words. We study its topological and combinatorial properties, in particular its homology groups,…
Algorithmic interpretability is necessary to build trust, ensure fairness, and track accountability. However, there is no existing formal measurement method for algorithmic interpretability. In this work, we build upon programming language…
Gradient descent during the learning process of a neural network can be subject to many instabilities. The spectral density of the Jacobian is a key component for analyzing stability. Following the works of Pennington et al., such Jacobians…
We introduce Thurstonian Boltzmann Machines (TBM), a unified architecture that can naturally incorporate a wide range of data inputs at the same time. Our motivation rests in the Thurstonian view that many discrete data types can be…
Functional brain networks exhibit topological structures that reflect neural organization; however, statistical comparison of these networks is challenging for several reasons. This paper introduces a topologically invariant permutation…
The Dense Associative Memory generalizes the Hopfield network by allowing for sharper interaction functions. This increases the capacity of the network as an autoassociative memory as nearby learned attractors will not interfere with one…
A measure of complexity based on a probabilistic description of physical systems is proposed. This measure incorporates the main features of the intuitive notion of such a magnitude. It can be applied to many physical situations and to…
We have recently shown that the statistical properties of goal directed reaching in human subjects depends on recent experience in a way that is consistent with the presence of adaptive Bayesian priors (Verstynen and Sabes, 2011). We also…
We introduce an asymmetric distance in the space of learning tasks, and a framework to compute their complexity. These concepts are foundational for the practice of transfer learning, whereby a parametric model is pre-trained for a task,…
Complex systems have become a popular lens for analyzing cities and complexity theory has many implications for urban performance and resilience. This paper develops a typology of measures and indicators for assessing the physical…
A recent breakthrough in biologically-plausible normative frameworks for dimensionality reduction is based upon the similarity matching cost function and the low-rank matrix approximation problem. Despite clear biological interpretation,…
The complex software systems developed nowadays require assessing their quality and proneness to errors. Reducing code complexity is a never-ending problem, especially in today's fast pace of software systems development. Therefore, the…