Related papers: Three tutorial lectures on entropy and counting
The Shannon entropy, and related quantities such as mutual information, can be used to quantify uncertainty and relevance. However, in practice, it can be difficult to compute these quantities for arbitrary probability distributions,…
We introduce the notions of topological entropy of a formal language and of a topological automaton. We show that the entropy function is surjective and bound the entropy of languages accepted by deterministic {\epsilon}-free push-down…
The first two lectures are devoted to describing the basic concepts of scattering theory in a very compressed way. A detailed presentation of the abstract part can be found in \cite{I} and numerous applications in \cite{RS} and \cite{Y2}.…
Classification is a machine learning method used in many practical applications: text mining, handwritten character recognition, face recognition, pattern classification, scene labeling, computer vision, natural langage processing. A…
The most fundamental properties of quantum entropy are derived by considering the union of two ensembles. We discuss the limits these properties put on an entropy measure and obtain that they uniquely determine the form of the entropy…
We develop a fine-scale local analysis of measure entropy and measure sequence entropy based on combinatorial independence. The concepts of measure IE-tuples and measure IN-tuples are introduced and studied in analogy with their…
In this article we continue to study the concept of entropy introduced in [4], [15]-[17]. We calculate entropy for a wider class of finite-dimensional operators in comparison with [15]. We also approximate the entropy of a unitary operator…
The notion of topological entropy can be conceptualized in terms of the number of forward trajectories that are distinguishable at resolution $\varepsilon$ within $T$ time units. It can then be formally defined as a limit of a limit…
Entropy and information can be considered dual: entropy is a measure of the subspace defined by the information constraining the given ambient space. Negative entropies, arising in na\"ive extensions of the definition of entropy from…
We examine the recently introduced NSB estimator of entropies of severely undersampled discrete variables and devise a procedure for calculating the involved integrals. We discover that the output of the estimator has a well defined limit…
We study reductions well suited to compare structures and classes of structures with respect to properties based on enumeration reducibility. We introduce the notion of a positive enumerable functor and study the relationship with…
Numerous entropy-type characteristics (functionals) generalizing R\'enyi entropy are widely used in mathematical statistics, physics, information theory, and signal processing for characterizing uncertainty in probability distributions and…
Shannon entropy is widely used to quantify the uncertainty of discrete random variables. But when normalized to the unit interval, as is often done in practice, it no longer conveys the alphabet sizes of the random variables being studied.…
We propose utilizing entropy as a diagnostic tool to distinguish between constant and dynamical dark energy models. Entropy, a measure of the system's disorder or information content, captures the complexity and evolution of the universe.…
We study the cross-entropy method (CEM) for the non-convex optimization of a continuous and parameterized objective function and introduce a differentiable variant that enables us to differentiate the output of CEM with respect to the…
This work is an enquiry into the circumstances under which entropy methods can give an answer to the questions of both quantum separability and classical correlations of a composite state. Several entropy functionals are employed to examine…
We define the notion of localizable property for a dynamical system. Then we survey three properties of complexity and relate how they are known to be typical among differentiable dynamical systems. These notions are the fast growth of the…
Environments and closures are two of the main ingredients of evaluation in lambda-calculus. A closure is a pair consisting of a lambda-term and an environment, whereas an environment is a list of lambda-terms assigned to free variables. In…
We study the entanglement entropy arising from coherent states and one--particle states. We show that it is possible to define a finite entanglement entropy by subtracting the vacuum entropy from that of the considered states, when the…
A central question for causal inference is to decide whether a set of correlations fit a given causal structure. In general, this decision problem is computationally infeasible and hence several approaches have emerged that look for…