Related papers: Shannon and R\'enyi entropy rates of stationary ve…
We study the entropy production of a system with a finite number of states connected by random transition rates. The stationary entropy production, driven out of equilibrium both by asymmetric transition rates and by an external probability…
Entropies must correspond to mean values for them to be measurable. The Shannon entropy corresponds to the weighted arithmetic mean, whereas the Renyi entropy corresponds to the exponential mean. These means refer to code lengths, which are…
Under a complex technical condition, similar to such used in extreme value theory, we find the rate q(\epsilon)^{-1} at which a stochastic process with stationary increments \xi should be sampled, for the sampled process \xi(\lfloor\cdot…
Stochastic blockmodels are generative network models where the vertices are separated into discrete groups, and the probability of an edge existing between two vertices is determined solely by their group membership. In this paper, we…
We consider the problem of approximating the stationary distribution of an ergodic Markov chain given a set of sampled transitions. Classical simulation-based approaches assume access to the underlying process so that trajectories of…
The seminal papers of Pickands [1,2] paved the way for a systematic study of high exceedance probabilities of both stationary and non-stationary Gaussian processes. Yet, in the vector-valued setting, due to the lack of key tools including…
In numerous applications data are observed at random times and an estimated graph of the spectral density may be relevant for characterizing and explaining phenomena. By using a wavelet analysis, one derives a nonparametric estimator of the…
Entropy rate of sequential data-streams naturally quantifies the complexity of the generative process. Thus entropy rate fluctuations could be used as a tool to recognize dynamical perturbations in signal sources, and could potentially be…
We prove the ergodic version of the quantum Stein's lemma which was conjectured by Hiai and Petz. The result provides an operational and statistical interpretation of the quantum relative entropy as a statistical measure of…
A generalization of the entropy production rate is proposed $\Pi_q$ in non-equilibrium systems by extending the formalism of classical stochastic thermodynamics to regimes with non-Gaussian fluctuations. Through the R\'enyi entropy $S_q$ ,…
Data partitioning that maximizes/minimizes the Shannon entropy, or more generally the R\'enyi entropy is a crucial subroutine in data compression, columnar storage, and cardinality estimation algorithms. These partition algorithms can be…
Shannon and Renyi entropies are quantitative measures of uncertainty in a data set. They are developed by Renyi in the context of entropy theory. These measures have been studied in the case of the multivariate t-distributions. We extend…
We consider a one-dimensional persisent random walk viewed as a deterministic process with a form of time reversal symmetry. Particle reservoirs placed at both ends of the system induce a density current which drives the system out of…
We calculate explicit formulae for the Shannon entropies of several families of tailored random graph ensembles for which no such formulae were as yet available, in leading orders in the system size. These include bipartite graph ensembles…
We examine the entropy of non-equilibrium stationary states of boundary driven totally asymmetric simple exclusion processes. As a consequence, we obtain that the Gibbs-Shannon entropy of the non equilibrium stationary state converges to…
The weak law of large numbers implies that, under mild assumptions on the source, the Renyi entropy per produced symbol converges (in probability) towards the Shannon entropy rate. This paper quantifies the speed of this convergence for…
We compute exact asymptotic results for the probability of the occurrence of large deviations of the largest (smallest) eigenvalue of random matrices belonging to the Gaussian orthogonal, unitary and symplectic ensembles. In particular, we…
Orey suggested the definition of some index for Gaussian processes with stationary increments which determines various properties of the sample paths of this process. We give an extension of the definition of the Orey index for a second…
The classical Gaussian ensembles of random matrices can be constructed by maximizing Boltzmann-Gibbs-Shannon's entropy, S_{BGS} = - \int d{\bf H} [P({\bf H})] \ln [P({\bf H})], with suitable constraints. Here we construct and analyze…
In this paper, we present a maximum likelihood estimation approach to determine the value vector in transformer models. We model the sequence of value vectors, key vectors, and the query vector as a sequence of Gaussian distributions. The…