Related papers: A local lemma via entropy compression
A well-known but rarely used approach to text categorization uses conditional entropy estimates computed using data compression tools. Text affinity scores derived from compressed sizes can be used for classification and ranking tasks, but…
The derivation of entropy for cluster methods is reformulated by constructing the probability of a given particle (spin) configuration as a self-consistent product of cluster configuration probabilities. This approach gives an insight into…
In this paper we introduce an alternative localization approach for binary classification that leads to a novel complexity measure: fixed points of the local empirical entropy. We show that this complexity measure gives a tight control over…
This paper presents an information theoretic approach to the concept of intelligence in the computational sense. We introduce a probabilistic framework from which computational intelligence is shown to be an entropy minimizing process at…
Model or variable selection is usually achieved through ranking models according to the increasing order of preference. One of methods is applying Kullback-Leibler distance or relative entropy as a selection criterion. Yet that will raise…
In this article, we study the pointwise asymptotic behavior of iterated convolutions on the one dimensional lattice Z. We generalize the so-called local limit theorem in probability theory to complex valued sequences. A sharp rate of…
We present herein a scheme by which to accurately evaluate the error exponents of a lossy data compression problem, which characterize average probabilities over a code ensemble of compression failure and success above or below a critical…
We develop tools for analyzing focused stochastic local search algorithms. These are algorithms which search a state space probabilistically by repeatedly selecting a constraint that is violated in the current state and moving to a random…
Given a probability measure $\mu$ over ${\mathbb R}^n$, it is often useful to approximate it by the convex combination of a small number of probability measures, such that each component is close to a product measure. Recently, Ronen Eldan…
We introduce a notion of a point-wise entropy of measures (i.e local entropy) called neutralized local entropy, and compare it with the Brin-Katok local entropy. We show that the neutralized local entropy coincides with Brin-Katok local…
This paper provides a general result on controlling local Rademacher complexities, which captures in an elegant form to relate the complexities with constraint on the expected norm to the corresponding ones with constraint on the empirical…
We introduce a new generalization of relative entropy to non-negative vectors with sums $\gt 1$. We show in a purely combinatorial setting, with no probabilistic considerations, that in the presence of linear constraints defining a convex…
We give a simple combinatorial algorithm to deterministically approximately count the number of satisfying assignments of general constraint satisfaction problems (CSPs). Suppose that the CSP has domain size $q=O(1)$, each constraint…
In 'An asymptotic result on compressed sensing matrices', a new construction for compressed sensing matrices using combinatorial design theory was introduced. In this paper, we use deterministic and probabilistic methods to analyse the…
Chang's lemma is a useful tool in additive combinatorics and the analysis of Boolean functions. Here we give an elementary proof using entropy. The constant we obtain is tight, and we give a slight improvement in the case where the…
We present a clustering method and provide a theoretical analysis and an explanation to a phenomenon encountered in the applied statistical literature since the 1990's. This phenomenon is the natural adaptability of the order when using a…
The Lovasz Local Lemma (LLL) is a powerful tool in probability theory to show the existence of combinatorial objects meeting a prescribed collection of "weakly dependent" criteria. We show that the LLL extends to a much more general…
In this paper, we develop a general approach to proving global and local uniform limit theorems for the Horvitz-Thompson empirical process arising from complex sampling designs. Global theorems such as Glivenko-Cantelli and Donsker…
Data used for analytics and machine learning often take the form of tables with categorical entries. We introduce a family of lossless compression algorithms for such data that proceed in four steps: $(i)$ Estimate latent variables…
Chang's Lemma is a widely employed result in additive combinatorics. It gives bounds on the dimension of the large spectrum of probability distributions on finite abelian groups. Recently, Bloom (2016) presented a powerful variant of…