相关论文: When do finite sample effects significantly affect…
Zipf's law is shown to arise as the variational solution of a problem formulated in Fisher's terms. An appropriate minimization process involving Fisher information and scale-invariance yields this universal rank distribution. As an example…
We introduce an epistemic information measure between two data streams, that we term $influence$. Closely related to transfer entropy, the measure must be estimated by epistemic agents with finite memory resources via sampling accessible…
Empirical relationships are derived for the expected sampling error of quantile estimations using Monte Carlo experiments for two frequency distributions frequently encountered in climate sciences. The relationships found are expressed as a…
We introduce a new measure of interdependence among the components of a random vector along the main diagonal of the vector copula, i.e. along the line $u_{1}=\ldots=u_{J}$, for $\left(u_{1},\ldots,u_{J}\right)\in\left[0,1\right]^{J}$. Our…
Entropy rate is a real valued functional on the space of discrete random sources which lacks a closed formula even for subclasses of sources which have intuitive parameterizations. A good way to overcome this problem is to examine its…
Here we present an analytic approximation for the entropy of floating-point numbers, along with bounds on the error of this approximation. It is well-known that the differential entropy is tightly linked to the discrete entropy of a…
Zipf's law implies the statistical distributions of hyperbolic type, which can describe the properties of stability and entropy loss in linguistics. We present the information theory from which follows that if the system is described by…
Diffusion models have excellent capacity to model complex distributions of natural data, which has made them a popular and effective choice for posterior sampling in imaging inverse problems. Existing methods can incorporate any measurement…
When fitting generalized linear mixed models (GLMMs), one important decision to make relates to the choice of the random effects distribution. As the random effects are unobserved, misspecification of this distribution is a real…
Importance sampling of target probability distributions belonging to a given convex class is considered. Motivated by previous results, the cost of importance sampling is quantified using the relative entropy of the target with respect to…
Zipf's law is a hallmark of several complex systems with a modular structure, such as books composed by words or genomes composed by genes. In these component systems, Zipf's law describes the empirical power law distribution of component…
The relationship between three probability distributions and their maximizable entropy forms is discussed without postulating entropy property. For this purpose, the entropy I is defined as a measure of uncertainty of the probability…
One of the defining traits of quantum mechanics is the uncertainty principle which was originally expressed in terms of the standard deviation of two observables. Alternatively, it can be formulated using entropic measures, and can also be…
If a measurement process is regarded as an irreversible process, then by Second law of thermodynamics the entropy should increase after any measurement process. By the same spirit a quantum system undergoing repeated measurement should show…
Causal effect estimation has been studied by many researchers when only observational data is available. Sound and complete algorithms have been developed for pointwise estimation of identifiable causal queries. For non-identifiable causal…
Estimating the entropy of a discrete random variable is a fundamental problem in information theory and related fields. This problem has many applications in various domains, including machine learning, statistics and data compression. Over…
We have presented a new axiomatic derivation of Shannon Entropy for a discrete probability distribution on the basis of the postulates of additivity and concavity of the entropy function.We have then modified shannon entropy to take account…
In QM/MM indirect free energy simulation, QM/MM corrections can be obtained from integration of partial derivatives of alchemical Hamiltonians or from perturbation-based estimators including free energy perturbation (FEP) and acceptance…
Power law distributions characterise several natural and social phenomena. The Zipf law for cities is one of those. The study views the question of whether that global regularity is independent of different spatial distributions of cities.…
Using symmetric boundary conditions at separated times, I show analytically that both the time ordering of (macroscopic) causality and the direction of entropy increase follow from these boundary conditions. In particular, when the…