Related papers: A Computable Measure of Algorithmic Probability by…
Kolmogorov complexity and algorithmic probability are defined only up to an additive resp. multiplicative constant, since their actual values depend on the choice of the universal reference computer. In this paper, we analyze a natural…
Problems of probabilistic inference and decision making under uncertainty commonly involve continuous random variables. Often these are discretized to a few points, to simplify assessments and computations. An alternative approximation is…
To deal with very large datasets a mini-batch version of the Monte Carlo Markov Chain Stochastic Approximation Expectation-Maximization algorithm for general latent variable models is proposed. For exponential models the algorithm is shown…
We derive sufficient conditions for a probability measure on a finite product space (a spin system) to satisfy a (modified) logarithmic Sobolev inequality. We establish these conditions for various examples, such as the (vertex-weighted)…
This paper studies several properties of channel codes that approach the fundamental limits of a given (discrete or Gaussian) memoryless channel with a non-vanishing probability of error. The output distribution induced by an…
Accurate approximation of probability measures is essential in numerical applications. This paper explores the quantization of probability measures using the maximum mean discrepancy (MMD) distance as a guiding metric. We first investigate…
The ability to find short representations, i.e. to compress data, is crucial for many intelligent systems. We present a theory of incremental compression showing that arbitrary data strings, that can be described by a set of features, can…
The advent of data science has spurred interest in estimating properties of distributions over large alphabets. Fundamental symmetric properties such as support size, support coverage, entropy, and proximity to uniformity, received most…
Sequence classification algorithms, such as SVM, require a definition of distance (similarity) measure between two sequences. A commonly used notion of similarity is the number of matches between $k$-mers ($k$-length subsequences) in the…
We consider the estimation of an i.i.d.\ random vector observed through a linear transform followed by a componentwise, probabilistic (possibly nonlinear) measurement channel. A novel algorithm, called generalized approximate message…
We comment on some conceptual and and technical problems related to computational mechanics, point out some errors in several papers, and straighten out some wrong priority claims. We present explicitly the correct algorithm for…
This article studies the asymptotic behaviors of nonparametric estimators of two overlapping measures, namely Pianka's and MacArthur-Levins measures. The plug-in principle and the method of kernel density estimation are used to estimate…
The paper develops new methods of non-parametric estimation a compound Poisson distribution. Such a problem arise, in particular, in the inference of a Levy process recorded at equidistant time intervals. Our key estimator is based on…
Sensitivity measures how much the output of an algorithm changes, in terms of Hamming distance, when part of the input is modified. While approximation algorithms with low sensitivity have been developed for many problems, no sensitivity…
The Robbins-Monro algorithm is a recursive, simulation-based stochastic procedure to approximate the zeros of a function that can be written as an expectation. It is known that under some technical assumptions, Gaussian limit distributions…
This paper provides a convergence analysis for generalized Hamiltonian Monte Carlo samplers, a family of Markov Chain Monte Carlo methods based on leapfrog integration of Hamiltonian dynamics and kinetic Langevin diffusion, that encompasses…
Computer science theory provides many different measures of complexity of a system including Kolmogorov complexity, logical depth, computational depth, and Levin complexity. However, these measures are all defined only for deterministic…
Despite the recent successes of probabilistic programming languages (PPLs) in AI applications, PPLs offer only limited support for random variables whose distributions combine discrete and continuous elements. We develop the notion of…
The aim of this paper is to approximate a finite-state Markov process by another process with fewer states, called herein the approximating process. The approximation problem is formulated using two different methods. The first method,…
Given matrices A and B and vectors a, b, c and d, all with non-negative entries, we consider the problem of computing min {c.x: x in Z^n_+, Ax > a, Bx < b, x < d}. We give a bicriteria-approximation algorithm that, given epsilon in (0, 1],…