Related papers: Rate distortion dimension and ergodic decompositio…
The generic behavior of quantum systems has long been of theoretical and practical interest. Any quantum process is represented by a sequence of quantum channels. We consider general ergodic sequences of stochastic channels with arbitrary…
Metric mean dimension is a dynamical counterpart of the box dimension in fractal geometry to characterize the topological complexity of infinite entropy systems. The classical variational principle states that topological entropy equals the…
Significant and persistent trajectory-to-trajectory variance are commonly observed in the particle tracking experiments, which have become a major challenge for the experiment data analysis. In this theoretical paper, we investigate the…
In this paper, we study rate-distortion theory for general sources with an emphasis on the existence of optimal reconstruction distributions on noncompact alphabets. Classical attainability results typically rely on compactness of the…
We extend the Rate-Distortion-Perception (RDP) framework to the R\'enyi information-theoretic regime, utilizing Sibson's $\alpha$-mutual information to characterize the fundamental limits under distortion and perception constraints. For…
In this paper we analyze the joint rate distortion function (RDF), for a tuple of correlated sources taking values in abstract alphabet spaces (i.e., continuous) subject to two individual distortion criteria. First, we derive structural…
In the last few years, various communication compression techniques have emerged as an indispensable tool helping to alleviate the communication bottleneck in distributed learning. However, despite the fact biased compressors often show…
We consider the dynamics of gradient descent (GD) in overparameterized single hidden layer neural networks with a squared loss function. Recently, it has been shown that, under some conditions, the parameter values obtained using GD achieve…
Suppose that we have $n$ agents and $n$ items which lie in a shared metric space. We would like to match the agents to items such that the total distance from agents to their matched items is as small as possible. However, instead of having…
This survey articles focuses on emerging connections between the fields of machine learning and data compression. While fundamental limits of classical (lossy) data compression are established using rate-distortion theory, the connections…
Recent results in quantization theory show that the mean-squared expected distortion can reach a rate of convergence of $\mathcal{O}(1/n)$, where $n$ is the sample size [see, e.g., IEEE Trans. Inform. Theory 60 (2014) 7279-7292 or Electron.…
Transformers achieve superior performance on many tasks, but impose heavy compute and memory requirements during inference. This inference can be made more efficient by partitioning the process across multiple devices, which, in turn,…
The study of finite approximations of probability measures has a long history. In (Xu and Berger, 2017), the authors focus on constrained finite approximations and, in particular, uniform ones in dimension $d=1$. The present paper gives an…
We consider the rate distortion problem with side information at the decoder posed and investigated by Wyner and Ziv. The rate distortion function indicating the trade-off between the rate on the data compression and the quality of data…
Peculiar velocities induce apparent line of sight displacements of galaxies in redshift space, distorting the pattern of clustering in the radial versus transverse directions. On large scales, the amplitude of the distortion yields a…
We develop a framework that we call compressive rate estimation. We assume that the composite channel gain matrix (i.e. the matrix of all channel gains between all network nodes) is compressible which means it can be approximated by a…
Gradient descent (GD) is a collection of continuous optimization methods that have achieved immeasurable success in practice. Owing to data science applications, GD with diminishing step sizes has become a prominent variant. While this…
An information theoretic formulation of the distributed averaging problem previously studied in computer science and control is presented. We assume a network with m nodes each observing a WGN source. The nodes communicate and perform local…
Over the last three decades, there has been a considerable effort within the applied probability community to develop techniques for bounding the convergence rates of general state space Markov chains. Most of these results assume the…
An expression for the dimensionless dissipation rate was derived from the Karman-Howarth equation by asymptotic expansion of the second- and third- order structure functions in powers of the inverse Reynolds number. The implications of the…