Related papers: Relative Resolution: An Analysis with the Kullback…
Recently, a novel type of a multiscale simulation, called Relative Resolution (RelRes), was introduced. In a single system, molecules switch their resolution in terms of their relative separation, with near neighbors interacting via…
Recently, we introduced Relative Resolution as a hybrid formalism for fluid mixtures [1]. The essence of this approach is that it switches molecular resolution in terms or relative separation: While nearest neighbors are characterized by a…
Selecting the optimal resolution for discretizing high-dimensional data is a central problem in physics and data analysis, particularly in unsupervised settings where the underlying distribution is unknown. The Relevance-Resolution…
The minimum Kullback entropy principle (mKE) is a useful tool to estimate quantum states and operations from incomplete data and prior information. In general, the solution of a mKE problem is analytically challenging and an approximate…
In this paper we review various information-theoretic characterizations of the approach to equilibrium in biological systems. The replicator equation, evolutionary game theory, Markov processes and chemical reaction networks all describe…
The maximum entropy formalism developed by Jaynes determines the relevant ensemble in nonequilibrium statistical mechanics by maximising the entropy functional subject to the constraints imposed by the available information. We present an…
We extend the application of the adaptive resolution technique (AdResS) to liquid systems composed of alkane chains of different lengths. The aim of the study is to develop and test the modifications of AdResS required in order to handle…
The entropy is one of the most applicable uncertainty measures in many statistical and en- gineering problems. In statistical literature, the entropy is used in calculation of the Kullback- Leibler (KL) information which is a powerful mean…
We give a new characterization of relative entropy, also known as the Kullback-Leibler divergence. We use a number of interesting categories related to probability theory. In particular, we consider a category FinStat where an object is a…
Entropy-like functionals on operator algebras have been studied since the pioneering work of von Neumann, Umegaki, Lindblad, and Lieb. The most well-known are the von Neumann entropy $trace (\rho\log \rho)$ and a generalization of the…
Uncertainty quantification in Large Language Models (LLMs) is crucial for applications where safety and reliability are important. In particular, uncertainty can be used to improve the trustworthiness of LLMs by detecting factually…
Formalising the confrontation of opinions (models) to observations (data) is the task of Inferential Statistics. Information Theory provides us with a basic functional, the relative entropy (or Kullback-Leibler divergence), an asymmetrical…
This paper proposes an impulse response modeling in presence of input and noisy output of a linear time-invariant (LTI) system. The approach utilizes Relative Entropy (RE) to choose the optimum impulse response estimate, optimum time delay…
We show here that molecular resolution is inherently hybrid in terms of relative separation: If molecules are close to each other, they must be characterized by a fine-grained (geometrically detailed) model, yet if molecules are far from…
The Kullback-Leibler divergence or relative entropy is an information-theoretic measure between statistical models that play an important role in measuring a distance between random variables. In the study of complex systems, random fields…
A range of nonlinear image reconstruction procedures based on extremizing the generalized Shannon entropy, Kullback-Leibler cross-entropy and Renyi information measures and proposed by the author in early papers is presented. The…
The necessary information to distinguish a local inhomogeneous mass density field from its spatial average on a compact domain of the universe can be measured by relative information entropy. The Kullback-Leibler (KL) formula arises very…
Many two-sample problems call for a comparison of two distributions from an exponential family. Density ratio estimation methods provide ways to solve such problems through direct estimation of the differences in natural parameters. The…
We examine the estimation of the Kullback-Leibler (KL) divergence and the use of the goodness-of-fit test for multivariate continuous distributions. Our starting point is the maximum entropy principle for Shannon entropy: among all…
We consider model-based reinforcement learning in finite Markov De- cision Processes (MDPs), focussing on so-called optimistic strategies. In MDPs, optimism can be implemented by carrying out extended value it- erations under a constraint…