Related papers: On typical encodings of multivariate ergodic sourc…
We prove that systems satisfying the specification property are saturated in the sense that the topological entropy of the set of generic points of any invariant measure is equal to the measure-theoretic entropy of the measure. We study…
Classical mathematical statistics deals with models that are parametrized by a Euclidean, i.e. finite dimensional, parameter. Quite often such models have been and still are chosen in practical situations for their mathematical simplicity…
We extend the study by Ornstein and Weiss on the asymptotic behavior of the normalized version of recurrence times and establish the large deviation property for a certain class of mixing processes. Further, an estimator for entropy based…
Positional encodings are a core part of transformer-based models, enabling processing of sequential data without recurrence. This paper presents a theoretical framework to analyze how various positional encoding methods, including…
We obtain an asymptotic normality result that reveals the precise asymptotic behavior of the maximum likelihood estimators of parameters for a very general class of linear mixed models containing cross random effects. In achieving the…
We study the existence, strong consistency and asymptotic normality of estimators obtained from estimating functions, that are p-dimensional martingale transforms. The problem is motivated by the analysis of evolutionary clustered data,…
Geometric Brownian motion is an exemplary stochastic processes obeying multiplicative noise, with widespread applications in several fields, e.g. in finance, in physics and biology. The definition of the process depends crucially on the…
Motivated from the fact that universal source coding on countably infinite alphabets is not feasible, this work introduces the notion of almost lossless source coding. Analog to the weak variable-length source coding problem studied by Han…
Bounds on the entropy of patterns of sequences generated by independently identically distributed (i.i.d.) sources are derived. A pattern is a sequence of indices that contains all consecutive integer indices in increasing order of first…
Using elementary methods, we prove that for a countable Markov chain $P$ of ergodic degree $d > 0$ the rate of convergence towards the stationary distribution is subgeometric of order $n^{-d}$, provided the initial distribution satisfies…
A new framework is introduced for examining and evaluating the fundamental limits of lossless data compression, that emphasizes genuinely non-asymptotic results. The {\em sample complexity} of compressing a given source is defined as the…
Gaussian quasi-likelihood estimation of the parameter $\theta$ in the square-root diffusion process is studied under high frequency sampling. Different from the previous study of Overbeck and Ryd\'{e}n(1998) under low-frequency sampling,…
Second order asymptotics of fixed-length source coding and intrinsic randomness is discussed with a constant error constraint. There was a difference between optimal rates of fixed-length source coding and intrinsic randomness, which never…
Consider $n$ i.i.d. random elements on $C[0,1]$. We show that, under an appropriate strengthening of the domain of attraction condition, natural estimators of the extreme-value index, which is now a continuous function, and the normalizing…
This paper investigates achievable information rates and error exponents of mismatched decoding when the channel belongs to the class of channels that are close to the decoding metric in terms of relative entropy. For both discrete- and…
The focus of this article is on entropy and Markov processes. We study the properties of functionals which are invariant with respect to monotonic transformations and analyze two invariant "additivity" properties: (i) existence of a…
Language models (LMs) are trained on billions of tokens in an attempt to recover the true language distribution. Still, vanilla random sampling from LMs yields low quality generations. Decoding algorithms attempt to restrict the LM…
We study spectrum of finite truncations of unbounded Jacobi matrices with periodically modulated entries. In particular, we show that under some hypotheses a sequence of properly normalized eigenvalue counting measures converge vaguely to…
We provide a novel achievability proof of the Slepian-Wolf theorem for i.i.d. sources over finite alphabets. We demonstrate that random codes that are linear over the real field achieve the classical Slepian-Wolf rate-region. For finite…
This paper studies the joint data and semantics lossy compression problem, i.e., an extension of the hidden lossy source coding problem that entails recovering both the hidden and observable sources. We aim to study the nonasymptotic and…