Related papers: General weighted information and relative informat…
Stimulated by the need of describing useful notions related to information measures, we introduce the `pdf-related distributions'. These are defined in terms of transformation of absolutely continuous random variables through their own…
An information-theoretic development is given for the problem of compound Poisson approximation, which parallels earlier treatments for Gaussian and Poisson approximation. Let $P_{S_n}$ be the distribution of a sum $S_n=\Sumn Y_i$ of…
A general random effects model is proposed that allows for continuous as well as discrete distributions of the responses. Responses can be unrestricted continuous, bounded continuous, binary, ordered categorical or given in the form of…
This presentation's Part 3 studies the evolutionary information processes and regularities of evolution dynamics, evaluated by an entropy functional (EF) of a random field (modeled by a diffusion information process) and an informational…
The dominant text classification studies focus on training classifiers using textual instances only or introducing external knowledge (e.g., hand-craft features and domain expert knowledge). In contrast, some corpus-level statistical…
Predictive models that generalize well under distributional shift are often desirable and sometimes crucial to building robust and reliable machine learning applications. We focus on distributional shift that arises in causal inference from…
The $n$-index R\'enyi mutual information and transfer entropy for the two-dimensional kinetic Ising model with arbitrary single-spin dynamics in the thermodynamic limit are derived as functions of thermodynamic quantities. By means of Monte…
The generalized inverse Gaussian-Poisson (GIGP) distribution proposed by Sichel in the 1970s has proved to be a flexible fitting tool for diverse frequency data, collectively described using the item production model. In this paper, we…
This article consists of an introduction to the theory of nonassociative geometric classical and quantum information flows defined by star products with R-flux deformations in string gravity. Corresponding nonassociative generalizations of…
Graph Neural Networks (GNNs) have exhibited impressive performance in many graph learning tasks. Nevertheless, the performance of GNNs can deteriorate when the input graph data suffer from weak information, i.e., incomplete structure,…
Quantifying synergy among stochastic variables is an important open problem in information theory. Information synergy occurs when multiple sources together predict an outcome variable better than the sum of single-source predictions. It is…
An information theory description of finite systems explicitly evolving in time is presented. We impose a MaxEnt variational principle on the Shannon entropy at a given time while the constraints are set at a former time. The resulting…
A framework for robust optimization under uncertainty based on the use of the generalized inverse distribution function (GIDF), also called quantile function, is here proposed. Compared to more classical approaches that rely on the usage of…
The Grade of Membership (GoM) model, which allows subjects to belong to multiple latent classes, is a powerful tool for inferring latent classes in categorical data. However, its application is limited to categorical data with nonnegative…
Extropy was introduced as a dual complement of the Shannon entropy. In this investigation, we consider failure extropy and its dynamic version. Various basic properties of these measures are presented. It is shown that the dynamic failure…
Graph neural networks (GNNs) have become powerful tools for processing graph-based information in various domains. A desirable property of GNNs is transferability, where a trained network can swap in information from a different graph…
An interactive stochastics, evaluated by an entropy functional (EF) of a random field and informational process' path functional (IPF), allows us modeling the evolutionary information processes and revealing regularities of evolution…
Entropic uncertainty relations are interesting in their own rights as well as for a lot of applications. Keeping this in mind, we try to make the corresponding inequalities as tight as possible. The use of parametrized entropies also allows…
Many social, technological, biological, and economical systems are best described by weighted networks, whose properties and dynamics depend not only on their structures but also on the connection weights among their nodes. However, most…
We propose a new framework for reasoning about information in complex systems. Our foundation is based on a variational extension of Shannon's information theory that takes into account the modeling power and computational constraints of…