Related papers: Why Occam's Razor
A powerful way to improve performance in machine learning is to construct an ensemble that combines the predictions of multiple models. Ensemble methods are often much more accurate and lower variance than the individual classifiers that…
As an approach to a Theory of Everything a framework for developing a coherent theory of mathematics and physics together is described. The main characteristic of such a theory is discussed: the theory must be valid and and sufficiently…
Quantum theory makes the most accurate empirical predictions and yet it lacks simple, comprehensible physical principles from which the theory can be uniquely derived. A broad class of probabilistic theories exist which all share some…
In physics, there is the prevailing intuition that we are part of a unique external world, and that the goal of physics is to understand and describe this world. This assumption of the fundamentality of objective reality is often seen as a…
Random matrix ensembles are introduced that respect the local tensor structure of Hamiltonians describing a chain of $n$ distinguishable spin-half particles with nearest-neighbour interactions. We prove a central limit theorem for the…
Ensemble-based methods are highly popular approaches that increase the accuracy of a decision by aggregating the opinions of individual voters. The common point is to maximize accuracy; however, a natural limitation occurs if incremental…
Promoting a theory with a finite number of terms into an effective field theory with an infinite number of terms worsens simplicity, predictability, falsifiability, and other attributes often favored in theory choice. However, the…
We formulate physically-motivated axioms for a physical theory which for systems with a finite number of degrees of freedom uniquely lead to Quantum Mechanics as the only nontrivial consistent theory. Complex numbers and the existence of…
The different interpretations of quantum mechanics yield the same experimental results, which may give the impression that the question of what interpretation is the true one, is a philosophical question, not a scientific one. But in this…
This book is concerned with the various aspects of hierarchical collective behaviour which is manifested by most complex systems in nature. From the many of the possible topics, we plan to present a selection of those that we think are…
Local explanation frameworks aim to rationalize particular decisions made by a black-box prediction model. Existing techniques are often restricted to a specific type of predictor or based on input saliency, which may be undesirably…
Solomonoff unified Occam's razor and Epicurus' principle of multiple explanations to one elegant, formal, universal theory of inductive inference, which initiated the field of algorithmic information theory. His central result is that the…
Understanding the behavior of learned classifiers is an important task, and various black-box explanations, logical reasoning approaches, and model-specific methods have been proposed. In this paper, we introduce probabilistic sufficient…
In many real world problems, features do not act alone but in combination with each other. For example, in genomics, diseases might not be caused by any single mutation but require the presence of multiple mutations. Prior work on feature…
Compositional generalization, the ability to recognize familiar parts in novel contexts, is a defining property of intelligent systems. Although modern models are trained on massive datasets, they still cover only a tiny fraction of the…
Social choice theory is the study of preference aggregation across a population, used both in mechanism design for human agents and in the democratic alignment of language models. In this study, we propose the representative social choice…
Component systems - ensembles of realizations built from a shared repertoire of modular parts - are ubiquitous in biological, ecological, technological, and socio-cultural domains. From genomes to texts, cities, and software, these systems…
Here we argue that the notion of falsifiability, a key concept in defining a valid scientific theory, can be quantified using Bayesian Model Selection, which is a standard tool in modern statistics. This relates falsifiability to the…
The ideas of Sensible Quantum Mechanics are expressed in lay terms for philosophers of consciousness and others. A framework is proposed and explained for the `psycho-physical-parallelism' between conscious experiences and the mathematical…
List learning is a variant of supervised classification where the learner outputs multiple plausible labels for each instance rather than just one. We investigate classical principles related to generalization within the context of list…