Related papers: Three tutorial lectures on entropy and counting

We translate the problem of calculating the entropy of a set of binary configurations/signals into a sequence of supervised classification tasks. Subsequently, one can use virtually any machine learning classification algorithm for…

The notion of metric entropy dimension is introduced to measure the complexity of entropy zero dynamical systems. For measure preserving systems, we define entropy dimension via the dimension of entropy generating sequences. This…

Entropy is a famous and well established concept in physics and engineering that can be used for explanation of basic fundamentals as well it finds applications in several areas, from quantum physics to astronomy, from network communication…

These are lecture notes written at the University of Zurich during spring 2014 and spring 2015. The first part of the notes gives an introduction to probability theory. It explains the notion of random events and random variables,…

Most of the existing classification methods are aimed at minimization of empirical risk (through some simple point-based error measured with loss function) with added regularization. We propose to approach this problem in a more information…

Given entropy's central role in multiple areas of physics and science, one important task is to develop a systematic and unifying approach to defining entropy. Games of chance become a natural candidate for characterising the uncertainty of…

Topological entropy is a widely studied indicator of chaos in topological dynamics. Here we give a generalized definition of topological entropy which may be applied to set-valued functions. We demonstrate that some of the well-known…

In these lecture notes, a selection of frequently required statistical tools will be introduced and illustrated. They allow to post-process data that stem from, e.g., large-scale numerical simulations (aka sequence of random experiments).…

Here we present an analytic approximation for the entropy of floating-point numbers, along with bounds on the error of this approximation. It is well-known that the differential entropy is tightly linked to the discrete entropy of a…

We introduce a concept for random tilings which, comprising the conventional one, is also applicable to tiling ensembles without height representation. In particular, we focus on the random tiling entropy as a function of the tile…

Some combinatorial properties of fixed boundary rhombus random tilings with octagonal symmetry are studied. A geometrical analysis of their configuration space is given as well as a description in terms of discrete dynamical systems, thus…

In the past several years, observational entropy has been developed as both a (time-dependent) quantum generalization of Boltzmann entropy, and as a rather general framework to encompass classical and quantum equilibrium and non-equilibrium…

A notion of entropy is introduced for causal fermion systems. This entropy is a measure of the state of disorder of a causal fermion system at a given time compared to the vacuum. The definition is given both in the finite and…

We study properties of popular near-uniform (Dirichlet) priors for learning undersampled probability distributions on discrete nonmetric spaces and show that they lead to disastrous results. However, an Occam-style phase space argument…

In these lectures we discuss some elementary concepts in connection with the theory of symmetric spaces applied to ensembles of random matrices. We review how the relationship between random matrix theory and symmetric spaces can be used in…

We review the plethora of uncertainty relations that appear in quantum mechanics and their nuances. We present both foundational applications, e.g. in understanding and defining complementarity, and practical applications, e.g. in quantum…

This is an introduction to measure theory, integration and function spaces, with all the needed preliminaries included, and with some applications included as well. We first discuss some basic motivations, coming from discrete probability,…

We present estimators for entropy and other functions of a discrete probability distribution when the data is a finite sample drawn from that probability distribution. In particular, for the case when the probability distribution is a joint…

A strengthened version of the central limit theorem for discrete random variables is established, relying only on information-theoretic tools and elementary arguments. It is shown that the relative entropy between the standardised sum of…

We propose a novel approach framed in terms of information theory and entropy to tackle the issue of conspiracy theories propagation. We start with the report of an event (such as 9/11 terroristic attack) represented as a series of…