Related papers: A General Notion of Useful Information
While various complexity measures for deep neural networks exist, specifying an appropriate measure capable of predicting and explaining generalization in deep networks has proven challenging. We propose Neural Complexity (NC), a…
Infinite sequences are of tremendous theoretical and practical importance, and in the Information Age sequences of 0s and 1s are of particular interest. Over the past century, the field of symbolic dynamics has developed to study sequences…
We introduce an algebraic framework for interacting quantum systems that enables studying complex phenomena, characterized by the coexistence and competition of various broken symmetry states of matter. The approach unveils the hidden unity…
Bennett's notion of depth is usually considered to describe the usefulness and internal organization of the information encoded into an object such as an infinite binary sequence. We consider a natural way to relativize the notion of depth…
We set up a parametrised monadic translation for a class of call-by-value functional languages, and prove a corresponding soundness theorem. We then present a series of concrete instantiations of our translation, demonstrating that a number…
We present an overview of some recent developments in the theory of generalized formal series, grounded in diffeological geometric framework. These constructions aim to offer new tools for understanding infinite-dimensional phenomena in…
We propose a more general definition of generic-case complexity, based on using a random process for generating inputs of an algorithm and using the time needed to generate an input as a way of measuring the size of that input.
Current learning algorithms face many difficulties in learning simple patterns and using them to learn more complex ones. They also require more examples than humans do to learn the same pattern, assuming no prior knowledge. In this paper,…
We introduce a new, systematic framework for visualizing information flow in deep networks. Specifically, given any trained deep convolutional network model and a given test image, our method produces a compact support in the image domain…
Model complexity is a fundamental problem in deep learning. In this paper we conduct a systematic overview of the latest studies on model complexity in deep learning. Model complexity of deep learning can be categorized into expressive…
Existing work on understanding deep learning often employs measures that compress all data-dependent information into a few numbers. In this work, we adopt a perspective based on the role of individual examples. We introduce a measure of…
Complex networks are universal, arising in fields as disparate as sociology, physics, and biology. In the past decade, extensive research into the properties and behaviors of complex systems has uncovered surprising commonalities among the…
Deep neural networks have seen enormous success in various real-world applications. Beyond their predictions as point estimates, increasing attention has been focused on quantifying the uncertainty of their predictions. In this review, we…
Information theory is a powerful framework for quantifying complexity, uncertainty, and dynamical structure in time-series data, with widespread applicability across disciplines such as physics, finance, and neuroscience. However, the…
This article summarises a Web-book on "Complexity" that was developed to introduce undergraduate students to interesting complex systems in the biological, physical and social sciences, and the common tools, principles and concepts used for…
This paper provides theoretical insights into why and how deep learning can generalize well, despite its large capacity, complexity, possible algorithmic instability, nonrobustness, and sharp minima, responding to an open question in the…
When we work with information from multiple sources, the formalism each employs to handle uncertainty may not be uniform. In order to be able to combine these knowledge bases of different formats, we need to first establish a common basis…
We provide a formal introduction into the classic theorems of general topology and its axiomatic foundations in set theory. In this second part we introduce the fundamental concepts of topological spaces, convergence, and continuity, as…
In many networks, including networks of protein-protein interactions, interdisciplinary collaboration networks, and semantic networks, connections are established between nodes with complementary rather than similar properties. While…
If a concept is not well defined, there are grounds for its abuse. This is particularly true of complexity, an inherently interdisciplinary concept that has penetrated very different fields of intellectual activity from physics to…