Related papers: Redundancy of information: lowering dimension
Despite many applications, dimensionality reduction in the $\ell_1$-norm is much less understood than in the Euclidean norm. We give two new oblivious dimensionality reduction techniques for the $\ell_1$-norm which improve exponentially…
Models often need to be constrained to a certain size for them to be considered interpretable. For example, a decision tree of depth 5 is much easier to understand than one of depth 50. Limiting model size, however, often reduces accuracy.…
We consider a stochastic version of the proximal point algorithm for optimization problems posed on a Hilbert space. A typical application of this is supervised learning. While the method is not new, it has not been extensively analyzed in…
We propose a "decomposition method" to prove non-asymptotic bound for the convergence of empirical measures in various dual norms. The main point is to show that if one measures convergence in duality with sufficiently regular observables,…
We present a theoretical framework that extends classical information theory to finite and structured systems by redefining redundancy as a fundamental property of information organization rather than inefficiency. In this framework,…
Every sufficiently big matrix with small spectral norm has a nearby low-rank matrix if the distance is measured in the maximum norm (Udell & Townsend, SIAM J Math Data Sci, 2019). We use the Hanson--Wright inequality to improve the estimate…
We prove results about subshifts with linear (word) complexity, meaning that $\limsup \frac{p(n)}{n} < \infty$, where for every $n$, $p(n)$ is the number of $n$-letter words appearing in sequences in the subshift. Denoting this limsup by…
Motivated by the problem of the definition of a distance between two sequences of characters, we investigate the so-called learning process of typical sequential data compression schemes. We focus on the problem of how a compression…
Consider a random matrix $H:\mathbb{R}^n\longrightarrow\mathbb{R}^m$. Let $D\geq2$ and let $\{W_l\}_{l=1}^{p}$ be a set of $k$-dimensional affine subspaces of $\mathbb{R}^n$. We ask what is the probability that for all $1\leq l\leq p$ and…
From the output produced by a memoryless deletion channel from a uniformly random input of known length $n$, one obtains a posterior distribution on the channel input. The difference between the Shannon entropy of this distribution and that…
Finding the underlying probability distributions of a set of observed sequences under the constraint that each sequence is generated i.i.d by a distinct distribution is considered. The number of distributions, and hence the number of…
High dimensional data can have a surprising property: pairs of data points may be easily separated from each other, or even from arbitrary subsets, with high probability using just simple linear classifiers. However, this is more of a rule…
We investigate how the Hausdorff dimensions of microsets are related to the dimensions of the original set. It is known that the maximal dimension of a microset is the Assouad dimension of the set. We prove that the lower dimension can…
The set of badly approximable $m \times n $ matrices is known to have Hausdorff dimension $mn $. Each such matrix comes with its own approximation constant $c$, and one can ask for the dimension of the set of badly approximable matrices…
We imagine an experiment on an unknown quantum mechanical system in which the system is prepared in various ways and a range of measurements are performed. For each measurement M and preparation rho the experimenter can determine, given…
Let $\Sigma$ be a countable alphabet. For $r\geq 1$, an infinite sequence $s$ with characters from $\Sigma$ is called $r$-quasi-regular, if for each $\sigma\in\Sigma$ the ratio of the longest to shortest interval between consecutive…
In this paper, we study a maximization problem on real sequences. More precisely, for a given sequence, we are interested in computing the supremum of the sequence and an index for which the associated term is maximal. We propose a general…
Motivated by applications to information retrieval, we study the lattice of antichains of finite intervals of a locally finite, totally ordered set. Intervals are ordered by reverse inclusion; the order between antichains is induced by the…
Kifer, Peres, and Weiss proved that there exists $c_0>0,$ such that $\dim \mu\leq 1-c_0$ for any probability measure $\mu$ which makes the digits of the continued fraction expansion i.i.d. random variables. In this paper we prove that…
Constant dimension codes, with a prescribed minimum distance, have found recently an application in network coding. All the codewords in such a code are subspaces of $\F_q^n$ with a given dimension. A computer search for large constant…