Related papers: Asymptotic Equipartition Property of Output when R…
Suppose a string $X_1^n=(X_1,X_2,...,X_n)$ generated by a memoryless source $(X_n)_{n\geq 1}$ with distribution $P$ is to be compressed with distortion no greater than $D\geq 0$, using a memoryless random codebook with distribution $Q$. The…
The asymptotic equipartition property (AEP) states that in the limit of a large number of independent and identically distributed (i.i.d.) random experiments, the output sequence is virtually certain to come from the typical set, each…
The classical asymptotic equipartition property is the statement that, in the limit of a large number of identical repetitions of a random experiment, the output sequence is virtually certain to come from the typical set, each member of…
This paper presents a rate-distortion theory for hierarchical networked data structures modelled as tree-indexed multitype process. To be specific, this paper gives a generalized Asymptotic Equipartition Property (AEP) for the Process. The…
Given an independent and identically distributed source $X = \{X_i \}_{i=1}^{\infty}$ with finite Shannon entropy or differential entropy (as the case may be) $H(X)$, the non-asymptotic equipartition property (NEP) with respect to $H(X)$ is…
The Asymptotic Equipartition Property (AEP) in information theory establishes that independent and identically distributed (i.i.d.) states behave in a way that is similar to uniform states. In particular, with appropriate smoothing, for…
We prove asymptotic equipartition properties for simple hierarchical structures (modelled as multitype Galton-Watson trees) and networked structures (modelled as randomly coloured random graphs). For example, for large $n$, a networked data…
Fluctuations in parameters that are typically treated as fixed play a crucial role in the behavior of complex systems. However, to date, we lack a general non-equilibrium thermodynamic treatment of such a complex system. In this Letter, to…
This paper studies properties of binary runlength-limited sequences with additional constraints on their Hamming weight and/or their number of runs of identical symbols. An algebraic and a probabilistic (entropic) characterization of the…
We present a development of parts of rate-distortion theory and pattern- matching algorithms for lossy data compression, centered around a lossy version of the Asymptotic Equipartition Property (AEP). This treatment closely parallels the…
Entropy production (EP) is known as a fundamental quantity for measuring the irreversibility of processes in thermal equilibrium and states far from equilibrium. In stochastic thermodynamics, the EP becomes more visible in terms of the…
We study asymptotic properties of expectation propagation (EP) -- a method for approximate inference originally developed in the field of machine learning. Applied to generalized linear models, EP iteratively computes a multivariate…
We prove a new asymptotic un-equipartition property for the perplexity of long texts generated by a language model and present supporting experimental evidence from open-source models. Specifically we show that the logarithmic perplexity of…
We propose a generalization of the asymptotic equipartition property to discrete sources with an ambiguous alphabet, and prove that it holds for irreducible stationary Markov sources with an arbitrary distinguishability relation. Our…
Approximate Bayesian computation allows for statistical analysis in models with intractable likelihoods. In this paper we consider the asymptotic behaviour of the posterior distribution obtained by this method. We give general results on…
We derive the asymptotic distribution of ordinal-pattern frequencies under weak dependence conditions and investigate the long-run covariance matrix not only analytically for moving-average, Gaussian, and the novel generalized coin-tossing…
Regularized system identification is the major advance in system identification in the last decade. Although many promising results have been achieved, it is far from complete and there are still many key problems to be solved. One of them…
The Symmetric Exclusion Process (SEP), in which particles hop symmetrically on a discrete line with hard-core constraints, is a paradigmatic model of subdiffusion in confined systems. This anomalous behavior is a direct consequence of…
We develop a new metric for quantifying end-to-end throughput in multihop wireless networks, which we term random access transport capacity, since the interference model presumes uncoordinated transmissions. The metric quantifies the…
In the theory of wireless communications, average performance measures (APMs) are widely utilized to quantify the performance gains/impairments in various fading environments under various scenarios, and to comprehend how the factors…