Related papers: A General Notion of Useful Information
We define the topological complexity sequence of a group as the sequence of topological complexities of its Milnor constructions. This sequence may be regarded as an intrinsic refinement of the topological complexity of a group and, unlike…
In this paper, we define the linear complexity for multidimensional sequences over finite fields, generalizing the one-dimensional case. We give some lower and upper bounds, valid with large probability, for the linear complexity and…
We initiate a study of the complexity of quantum field theories (QFTs) by proposing a measure of information contained in a QFT and its observables. We show that from minimal assertions, one is naturally led to measure complexity by two…
This paper aims to define, quantify, and analyze the feature complexity that is learned by a DNN. We propose a generic definition for the feature complexity. Given the feature of a certain layer in the DNN, our method disentangles feature…
We introduce a unified theoretical framework for the rigorous analysis and systematic construction of deep neural networks (DNNs). This framework addresses a gap in existing theory by explicitly modeling the structure of tensor operations…
A substantial volume of research has been devoted to studies of community structure in networks, but communities are not the only possible form of large-scale network structure. Here we describe a broad extension of community structure that…
The complexity of knowledge production on complex systems is well-known, but there still lacks knowledge framework that would both account for a certain structure of knowledge production at an epistemological level and be directly…
The framework of generalized probabilistic theories is a powerful tool for studying the foundations of quantum physics. It provides the basis for a variety of recent findings that significantly improve our understanding of the rich physical…
We discuss the role of propositions, truth, context and observers in scientific theories. We introduce the concept of generalized proposition and use it to define an algorithm for the classification of any scientific theory. The algorithm…
There is a significant lack of unified approaches to building generally intelligent machines. The majority of current artificial intelligence research operates within a very narrow field of focus, frequently without considering the…
Networks are a fundamental tool for understanding and modeling complex systems in physics, biology, neuroscience, engineering, and social science. Many networks are known to exhibit rich, lower-order connectivity patterns that can be…
What is a complex network? How do we characterize complex networks? Which systems can be studied from a network approach? In this text, we motivate the use of complex networks to study and understand a broad panoply of systems, ranging from…
Centrality descriptors are widely used to rank nodes according to specific concept(s) of importance. Despite the large number of centrality measures available nowadays, it is still poorly understood how to identify the node which can be…
As a measure for the centrality of a point in a set of multivariate data, statistical depth functions play important roles in multivariate analysis, because one may conveniently construct descriptive as well as inferential procedures…
We introduce a new notion of complexity of functions and we show that it has the following properties: (i) it governs a PAC Bayes-like generalization bound, (ii) for neural networks it relates to natural notions of complexity of functions…
This technical report presents a general framework for parsing a variety of grammar formalisms. We develop a grammar formalism, called an Abstract Grammar, which is general enough to represent grammars at many levels of the hierarchy,…
We present a general form of attribute exploration, a knowledge completion algorithm from Formal Concept Analysis. The aim of our presentation is not only to extend the applicability of attribute exploration by a general description. It may…
Observations on the past provide some hints about what will happen in the future, and this can be quantified using information theory. The ``predictive information'' defined in this way has connections to measures of complexity that have…
Managing inputs that are novel, unknown, or out-of-distribution is critical as an agent moves from the lab to the open world. Novelty-related problems include being tolerant to novel perturbations of the normal input, detecting when the…
We survey current developments in the approximation theory of sequence modelling in machine learning. Particular emphasis is placed on classifying existing results for various model architectures through the lens of classical approximation…