相关论文: Range-Renewal Speed and Entropy for I.I.D Models
We establish upper and lower bounds for the metric entropy and bracketing entropy of the class of $d$-dimensional bounded monotonic functions under $L^p$ norms. It is interesting to see that both the metric entropy and bracketing entropy…
In [1] it is shown that recurrent neural networks (RNNs) can learn - in a metric entropy optimal manner - discrete time, linear time-invariant (LTI) systems. This is effected by comparing the number of bits needed to encode the…
In this paper, we introduce the notion of recurrence entropy in the context of nonlinear control systems. A set is said to be ($\tau$-)recurrent if every trajectory that starts in the set returns to it (within at most $\tau$ units of time).…
The paper presents a lightweight discussion of relations between energy, entropy, information, and intelligence, based on an analysis of the energy needed for computation.
For a d-dimensional cellular automaton with d $\ge$ 1 we introduce a rescaled entropy which estimates the growth rate of the entropy at small scales by generalizing previous approaches [1, 9]. We also define a notion of Lyapunov exponent…
For sensory networks, we determine the rate with which they acquire information about the changing external conditions. Comparing this rate with the thermodynamic entropy production that quantifies the cost of maintaining the network, we…
In the context of state estimation under communication constraints, several notions of dynamical entropy play a fundamental role, among them: topological entropy and restoration entropy. In this paper, we present a theorem which…
The time variation of entropy, as an alternative to the variance, is proposed as a measure of the diffusion rate. It is shown that for linear and time-translationally invariant systems having a large-time limit for the density, at large…
We suggest to construct infinite stochastic binary sequences by associating one of the two symbols of the sequence with the renewal times of an underlying renewal process. Focusing on stationary binary sequences corresponding to delayed…
This short note presents the derivation of a new {\it a priori} estimate for the Oldroyd-B model. Such an estimate may provide useful information when investigating the long-time behaviour of macro-macro models, and the stability of…
The Renyi entropy is a generalization of the usual concept of entropy which depends on a parameter q. In fact, Renyi entropy is closely related to free energy. Suppose we start with a system in thermal equilibrium and then suddenly divide…
We consider renewal-type processes whose positive inter-renewal times may be dependent, non-identically distributed, and may have mixed distributions. We introduce a generalised intensity measure extending the classical hazard-rate…
Renewal processes are broadly used to model stochastic behavior consisting of isolated events separated by periods of quiescence, whose durations are specified by a given probability law. Here, we identify the minimal sufficient statistic…
Consider a finite renewal process in the sense that interrenewal times are positive i.i.d. variables and the total number of renewals is a random variable, independent of interrenewal times. A finite point process can be obtained by…
The entropy power inequality for independent random vectors is a foundational result of information theory, with deep connections to probability and geometric functional analysis. Several extensions of the entropy power inequality have been…
The ability to continuously learn remains elusive for deep learning models. Unlike humans, models cannot accumulate knowledge in their weights when learning new tasks, mainly due to an excess of plasticity and the low incentive to reuse…
Defining the density flow of perturbations moving at a given speed for cellular automata, we establish equalities and inequalities between the measurable entropy of a cellular automaton and the measurable entropy of its associated shift.
We derive generic upper bounds on the rate of purity change and entropy increase for open quantum systems. These bounds depend solely on the generators of the nonunitary dynamics and are independent of the particular states of the systems.…
The ability to estimate the rate of convergence for the distributions of regenerative processes is in great demand. These processes are often encountered in queuing theory and in related problems. In some papers on regenerative processes,…
In this work we investigate the information loss in (nonlinear) dynamical input-output systems and provide some general results. In particular, we present an upper bound on the information loss rate, defined as the (non-negative) difference…