Related papers: Algorithmic information and simplicity in statisti…
In this paper, we generalize and improve some fundamental concentration inequalities using information on the random variables' higher moments. In particular, we improve the classical Hoeffding's and Bennett's inequalities for the case…
A general condition determining the optimal performance of a complex system has not yet been found and the possibility of its existence is unknown. To contribute in this direction, an optimization algorithm as a complex system is presented.…
The possibility of calculation of the conditional and unconditional complexity of description of information objects in the algorithmic theory of information is connected with the limitations for the set of the used languages of programming…
"Help bits" are some limited trusted information about an instance or instances of a computational problem that may reduce the computational complexity of solving that instance or instances. In this paper, we study the value of help bits in…
A central question in computer science and statistics is whether efficient algorithms can achieve the information-theoretic limits of statistical problems. Many computational-statistical tradeoffs have been shown under average-case…
Missing data are frequently encountered in high-dimensional problems, but they are usually difficult to deal with using standard algorithms, such as the expectation-maximization (EM) algorithm and its variants. To tackle this difficulty,…
The conditional entropy power inequality is a fundamental inequality in information theory, stating that the conditional entropy of the sum of two conditionally independent vector-valued random variables each with an assigned conditional…
We prove oracle inequalities for a penalized log-likelihood criterion that hold even if the data are not independent and not stationary, based on a martingale approach. The assumptions are checked for various contexts: density estimation…
Global changes of states are of crucial importance in optimization algorithms. We review some heuristic algorithms in which global updates are realized by a sort of real-space renormalization group transformation. Emphasis is on the…
Given a Probabilistic Finite Automata (PFA), a set of states S, and an error threshold e > 0, our algorithm approximates the infimum probability (quantifying over all infinite words) that the automata reaches S. Our result contrasts with…
Randomness plays a central rol in the quantum mechanical description of our interactions. We review the relationship between the violation of Bell inequalities, non signaling and randomness. We discuss the challenge in defining a random…
It is argued that the nature of probability is essentially informational rather than physical and that quantum mechanical predictions should be viewed as logical inferences made on the basis of the information content of a given…
Landauer's Principle that information loss from a computation implies entropy increase can be rigorously proved from mathematical physics. However, carefully examining its detailed formulation reveals that the traditional identification of…
I propose a "quantum annealing" heuristic for the problem of combinatorial search among a frustrated set of states characterized by a cost function to be minimized. The algorithm is probabilistic, with postselection of the measurement…
In this paper, we consider several types of information and methods of combination associated with incomplete probabilistic systems. We discriminate between 'a priori' and evidential information. The former one is a description of the whole…
We propose that a quantum particle in a potential in one space dimension can be described by a probabilistic cellular automaton. While the simple updating rule of the automaton is deterministic, the probabilistic description is introduced…
We present a new algorithm for exactly solving decision making problems represented as influence diagrams. We do not require the usual assumptions of no forgetting and regularity; this allows us to solve problems with simultaneous decisions…
We investigate the in-distribution generalization of machine learning algorithms. We depart from traditional complexity-based approaches by analyzing information-theoretic bounds that quantify the dependence between a learning algorithm and…
When designing optimal controllers for any system, it is often the case that the true state of the system is unknown to the controller, for example due to noisy measurements or partially observable states. Incomplete state information must…
Algorithmic statistics studies explanations of observed data that are good in the algorithmic sense: an explanation should be simple i.e. should have small Kolmogorov complexity and capture all the algorithmically discoverable regularities…