Related papers: Lecture Notes on Algorithmic Information Theory
Open world Machine Learning (OWML) aims to develop intelligent systems capable of recognizing known categories, rejecting unknown samples, and continually learning from novel information. Despite significant progress in open set…
We introduce \emph{Information Topology}: a framework that unifies information theory and algebraic topology by treating \emph{cycle closure} as the primitive operation of inference. The starting point is the \emph{dot-cycle dichotomy},…
Computational complexity is a core theory of computer science, which dictates the degree of difficulty of computation. There are many problems with high complexity that we have to deal, which is especially true for AI. This raises a big…
The outcome of all time series cannot be forecast, e.g. the flipping of a fair coin. Others, like the repeated {01} sequence {010101...} can be forecast exactly. Algorithmic information theory can provide a measure of forecastability that…
The notion of an individual random sequence goes back to von Mises. We describe the evolution of this notion, especially the use of martingales (suggested by Ville), and the development of algorithmic information theory in 1960s and 1970s…
Our capacity to process information depends on the computational power at our disposal. Information theory captures our ability to distinguish states or communicate messages when it is unconstrained with unrivaled beauty and elegance. For…
This paper fills a gap in our understanding of the interaction between information and computation. It unifies other approaches to measuring information like Kolmogorov complexity and Shannon information. We define a theory about…
This is a course on the theory of quantum computing. It consists of 16 lessons, each with a video and written component, covering the basics of quantum information, quantum algorithms (including query algorithms, Shor's algorithm for…
Some Goedel centenary reflections on whether incompleteness is really serious, and whether mathematics should be done somewhat differently, based on using algorithmic complexity measured in bits of information. [Enriques lecture given…
The study of the fundamental limits of information systems is a central theme in information theory. Both the traditional analytical approach and the recently proposed computational approach have significant limitations, where the former is…
The progress of machine learning over the past decade is undeniable. In retrospect, it is both remarkable and unsettling that this progress was achievable with little to no rigorous theory to guide experimentation. Despite this fact,…
In the paper we present results to develop an irreducible theory of complex systems in terms of self-organization processes of prime integer relations. Based on the integers and controlled by arithmetic only the self-organization processes…
Chaitin's work, in its depth and breadth, encompasses many areas of scientific and philosophical interest. It helped establish the accepted mathematical concept of randomness, which in turn is the basis of tools that I have developed to…
This is the text of my habilitation thesis defended at the \'Ecole Normale Sup\'erieure. The topological string presents an arena in which many features of string theory proper, such as the interplay between worldsheet and target space…
This note is intended to foster a discussion about the extent to which typical problems arising in quantum information theory are algorithmically decidable (in principle rather than in practice). Various problems in the context of…
Claude Shannons information theory (1949) has had a revolutionary impact on communication science. A crucial property of his framework is that it decouples the meaning of a message from the mechanistic details from the actual communication…
We introduce a framework uniting algorithmic randomness with exchangeable credences to address foundational questions in philosophy of probability and philosophy of science. To demonstrate its power, we show how one might use the framework…
The probability distribution P from which the history of our universe is sampled represents a theory of everything or TOE. We assume P is formally describable. Since most (uncountably many) distributions are not, this imposes a strong…
Theoretical computer science discusses foundational issues about computations. It asks and answers questions such as "What is a computation?", "What is computable?", "What is efficiently computable?","What is information?", "What is…
These notes were compiled as lecture notes for a course developed and taught at the University of the Southern California. They should be accessible to a typical engineering graduate student with a strong background in Applied Mathematics.…