Related papers: Lecture Notes on Algorithmic Information Theory
These are notes for the course CS-172 I first taught in the Fall 1986 at UC Berkeley and subsequently at Boston University. The goal was to introduce the undergraduates to basic concepts of Theory of Computation and to provoke their…
Despite its great success, machine learning can have its limits when dealing with insufficient training data. A potential solution is the additional integration of prior knowledge into the training process which leads to the notion of…
We extend in a natural way the operation of Turing machines to infinite ordinal time, and investigate the resulting supertask theory of computability and decidability on the reals. The resulting computability theory leads to a notion of…
We formulate meta learning using information theoretic concepts; namely, mutual information and the information bottleneck. The idea is to learn a stochastic representation or encoding of the task description, given by a training set, that…
This research is about operational- and complexity-oriented aspects of classical foundations of computability theory. The approach is to re-examine some classical theorems and constructions, but with new criteria for success that are…
The notion of probability plays an important role in almost all areas of science and technology. In modern mathematics, however, probability theory means nothing other than measure theory, and the operational characterization of the notion…
This article is a short review on the concept of information. We show the strong relation between Information Theory and Physics, beginning by the concept of bit and its representation with classical physical systems, and then going to the…
A practical notation can convey valuable intuitions and concisely express new ideas. Information theory is of importance to machine learning, but the notation for information-theoretic quantities is sometimes opaque. We propose a practical…
The education system for students in physics suffers (worldwide) from the absence of a deep course in probability and randomness. This is the real problem for students interested in quantum information theory, quantum optics, and quantum…
We present an information-theoretic interpretation of quantum formalism based on a Bayesian framework and devoid of any extra axiom or principle. Quantum information is construed as a technique for analyzing a logical system subject to…
This is an extended and corrected version of lecture notes originally written for a one semester course at Leibniz University Hannover. The main aim of the notes is to give an introduction to the mathematical methods used in describing…
We introduce and study several notions of computability-theoretic reducibility between subsets of $\omega$ that are "robust" in the sense that if only partial information is available about the oracle, then partial information can be…
As the second book in the Anyone Can Code series, Algorithmic Thinking focuses on the logic behind computer programming and software design. With a data-centred approach, it starts with simple algorithms that work on simple data items and…
Despite ongoing calls for inclusive and culturally responsive pedagogy in computing education, the teaching of algorithms remains largely decontextualized. Foundational computer science courses often present algorithmic thinking as purely…
At this point in time, two major areas of physics, statistical mechanics and quantum mechanics, rest on the foundations of probability and entropy. The last century saw several significant fundamental advances in our understanding of the…
The unwavering success of deep learning in the past decade led to the increasing prevalence of deep learning methods in various application fields. However, the downsides of deep learning, most prominently its lack of trustworthiness, may…
These notes are based on a lecture delivered by NC on March 2021, as part of an advanced course in Princeton University on the mathematical understanding of deep learning. They present a theory (developed by NC, NR and collaborators) of…
We introduce the notion of a reproducible algorithm in the context of learning. A reproducible learning algorithm is resilient to variations in its samples -- with high probability, it returns the exact same output when run on two samples…
These are the lecture notes of the master's course "Quantum Computing", taught at Chalmers University of Technology every fall since 2020, with participation of students from RWTH Aachen and Delft University of Technology. The aim of this…
These lecture notes provide an introduction to free probability theory, with a focus on tools and techniques useful in the study of large random matrices. Topics include freeness, free cumulants, additive and multiplicative free…