Related papers: On the Information Theoretic Distance Measures and…

Complex systems often exhibit multiple levels of organization covering a wide range of physical scales, so the study of the hierarchical decomposition of their structure and function is frequently convenient. To better understand this…

A prediction makes a claim about a system's future given knowledge of its past. A retrodiction makes a claim about its past given knowledge of its future. The bidirectional machine is an ambidextrous hidden Markov chain that does both…

Information theory is an outstanding framework to measure uncertainty, dependence and relevance in data and systems. It has several desirable properties for real world applications: it naturally deals with multivariate data, it can handle…

We extend Elitzur's theorem to systems with symmetries intermediate between global and local. In general, our theorem formalizes the idea of {\it dimensional reduction}. We apply the results of this generalization to many systems that are…

We define discrete Hamiltonian systems in the framework of discrete embeddings. An explicit comparison with previous attempts is given. We then solve the discrete Helmholtz's inverse problem for the discrete calculus of variation in the…

Generalization error bounds are essential to understanding machine learning algorithms. This paper presents novel expected generalization error upper bounds based on the average joint distribution between the output hypothesis and each…

We remind and emphasize the connections and differences between Kirchhoff and Born modelling. We discuss how they lead to a general expression for the conversion of a velocity model perturbation into a reflectivity through the "generalized…

In designing an intelligent system that must be able to explain its reasoning to a human user, or to provide generalizations that the human user finds reasonable, it may be useful to take into consideration psychological data on what types…

Many machine learning methods assume that the training and test data follow the same distribution. However, in the real world, this assumption is very often violated. In particular, the phenomenon that the marginal distribution of the data…

The notion of a generalized Lie bialgebroid (a generalization of the notion of a Lie bialgebroid) is introduced in such a way that a Jacobi manifold has associated a canonical generalized Lie bialgebroid. As a kind of converse, we prove…

We examine the relationship between the mutual information between the output model and the empirical sample and the generalization of the algorithm in the context of stochastic convex optimization. Despite increasing interest in…

Widespread use of the Internet and social networks invokes the generation of big data, which is proving to be useful in a number of applications. To deal with explosively growing amounts of data, data analytics has emerged as a critical…

We derive information-theoretic generalization bounds for supervised learning algorithms based on the information contained in predictions rather than in the output of the training algorithm. These bounds improve over the existing…

Comparing networks is essential for a number of downstream tasks, from clustering to anomaly detection. Despite higher-order interactions being critical for understanding the dynamics of complex systems, traditional approaches for network…

We introduce a hierarchical classification of theories that describe systems with fundamentally limited information content. This property is introduced in an operational way and gives rise to the existence of mutually complementary…

In this paper we analyse the Waveholtz method, a time-domain iterative method for solving the Helmholtz iteration, in the constant-coefficient case in all of $\mathbb{R}^d$. We show that the difference between a Waveholtz iterate and the…

This paper is devoted to the mathematical study of some divergences based on the mutual information well-suited to categorical random vectors. These divergences are generalizations of the "entropy distance" and "information distance". Their…

We consider a hybrid bimetric model where, in addition to the ordinary metric tensor that determines geometry, an informational metric is introduced to describe the reference frame of an observer. We note that the local information metric…

Modern machine learning increasingly leverages the insight that high-dimensional data often lie near low-dimensional, non-linear manifolds, an idea known as the manifold hypothesis. By explicitly modeling the geometric structure of data…

We investigate the in-distribution generalization of machine learning algorithms. We depart from traditional complexity-based approaches by analyzing information-theoretic bounds that quantify the dependence between a learning algorithm and…