Related papers: A brief tutorial on information theory
We introduce a new kind of Information Theory. From a finite number of local, noisy comparisons, we want to design a robust filter such that the outcome is a high ranking number, Both analytical and numerical results are encouraging and we…
An article based on a four-lecture introductory minicourse on minimal surface theory given at the 2013 summer program of the Institute for Advanced Study and the Park City Mathematics Institute.
Computational intelligence is broadly defined as biologically-inspired computing. Usually, inspiration is drawn from neural systems. This article shows how to analyze neural systems using information theory to obtain constraints that help…
Quantum information is an emerging field which has attracted a lot of attention in the last fifteen years or so. It is a broad subject which covers from the most applied questions (e.g., how to build quantum computers or secure…
This series of introductory lectures consists of two parts. In the first part, I rapidly review the basic notions of quantum physics and many primitives of quantum information (i.e. notions that one must be somehow familiar with in the…
This paper introduces an information theoretic co-training objective for unsupervised learning. We consider the problem of predicting the future. Rather than predict future sensations (image pixels or sound waves) we predict "hypotheses" to…
Information theory, i.e. the mathematical analysis of information and of its processing, has become a tenet of modern science; yet, its use in real-world studies is usually hindered by its computational complexity, the lack of coherent…
Stochastic networks represent very important subject of research because they have been found in almost all branches of modern science, including also sociology and economy. We provide a information theory point of view, mostly based on its…
In this article a notion of information is presented which stresses the contextuality of quantum objects and their measurement. Mathematically this is reached by a quantification of the quantum mechanical surplus knowledge which has been…
This work is a discussion on the concept of information. We define here information as an abstraction that is able to be copied. We consider the connection between the process of copying information in quantum systems and the emergence of…
These are the notes for two lectures delivered at the Les Houches summer school Mathematical Statistical Mechanics, held in July 2005. I review some basic notions on sparse graph error correcting codes with emphasis on `modern' aspects,…
This text is a slightly edited version of lecture notes for a course I gave at ETH, during the Summer term 2001, to undergraduate Mathematics and Physics students. It covers a few selected topics from perturbation theory at an introductory…
As a result of the capabilities of quantum information, the science of quantum information processing is now a prospering, interdisciplinary field focused on better understanding the possibilities and limitations of the underlying theory,…
We propose a system of information-theoretic axioms from which we derive the formalism of quantum theory. Part I is devoted to the conceptual foundations of the information-theoretic approach. We argue that this approach belongs to the…
Lecture notes from the course given by Professor Sara A. Solla at the Les Houches summer school on "Statistical physics of Machine Learning". The notes discuss neural information processing through the lens of Statistical Physics. Contents…
These are the notes for a set of lectures delivered by the two authors at the Les Houches Summer School on `Complex Systems' in July 2006. They provide an introduction to the basic concepts in modern (probabilistic) coding theory,…
This book is intended for teaching Signal Analysis methods and Inverse Problems theory. It is completely open access and will remain free. It is currently illustrated with examples that we have actually encountered in geophysics, but will…
Information Theory concepts and methodologies conform the background of how communication systems are studied and understood. They are mainly focused on the source-channel-receiver problem and on the asymptotic limits of accuracy and…
This text was published in the Gazette de la Soci\'et\'e Math\'ematique de France in October 2023. It is an introduction to the theory of motives, from its sources to its more modern developments.
The principles of statistical mechanics and information theory play an important role in learning and have inspired both theory and the design of numerous machine learning algorithms. The new aspect in this paper is a focus on integrating…