Related papers: On Enumerating Distributions for Associated Vector…
We study the distribution regression problem assuming the distribution of distributions has a doubling measure larger than one. First, we explore the geometry of any distributions that has doubling measure larger than one and build a small…
A problem of improving the accuracy of nonparametric entropy estimation for a stationary ergodic process is considered. New weak metrics are introduced and relations between metrics, measures, and entropy are discussed. Based on weak…
We consider density estimators based on the nearest neighbors method applied to discrete point distibutions in spaces of arbitrary dimensionality. If the density is constant, the volume of a hypersphere centered at a random location is…
We review and generalize the recently introduced framework of entropy vectors for detecting and quantifying genuine multipartite entanglement in high dimensional multicomponent quantum systems. We show that these ideas can be extended to…
We investigate correlations in information carriers, e.g. texts and pieces of music, which are represented by strings of letters. For information carrying strings generated by one source (i.e. a novel or a piece of music) we find…
Estimating information-theoretic quantities such as entropy and mutual information is central to many problems in statistics and machine learning, but challenging in high dimensions. This paper presents estimators of entropy via inference…
Numerous entropy-type characteristics (functionals) generalizing R\'enyi entropy are widely used in mathematical statistics, physics, information theory, and signal processing for characterizing uncertainty in probability distributions and…
In its continuous version, the entropy functional measuring the information content of a given probability density may be plagued by a "measure" problem that results from improper weighting of phase space. This issue is addressed…
We propose a nonparametric estimator of multivariate joint entropy based on partitioned sample spacing (PSS). The method extends univariate spacing ideas to $\mathbb{R}^{d}$ by partitioning into localized cells and aggregating within-cell…
The metric dimension of non-component graph, associated to a finite vector space, is determined. It is proved that the exchange property holds for resolving sets of the graph, except a special case. Some results are also related to an…
Consider a finite set of sources, each producing i.i.d. observations that follow a unique probability distribution on a finite alphabet. We study the problem of matching a finite set of observed sequences to the set of sources under the…
Distributed representations of words learned from text have proved to be successful in various natural language processing tasks in recent times. While some methods represent words as vectors computed from text using predictive model…
We present a simple combinatorial framework for establishing approximate tensorization of variance and entropy in the setting of spin systems (a.k.a. undirected graphical models) based on balanced separators of the underlying graph. Such…
Quantitative estimates are derived, on the whole space, for the relative entropy between the joint law of random interacting particles and the tensorized law at the limiting systeme. The developed method combines the relative entropy method…
We propose a neural embedding algorithm called Network Vector, which learns distributed representations of nodes and the entire networks simultaneously. By embedding networks in a low-dimensional space, the algorithm allows us to compare…
We study the possibility that black hole entropy be identified as entropy of entanglement across the horizon of the vacuum of a quantum field in the presence of the black hole. We argue that a recent proposal for computing entanglement…
Motivated by real-world applications such as the allocation of public housing, we examine the problem of assigning a group of agents to vertices (e.g., spatial locations) of a network so that the diversity level is maximized. Specifically,…
Probability distributions defined on the half space are known to be quite different from those in the full space. Here, a nonextensive entropic treatment is presented for the half space in an analytic and self-consistent way. In this…
This work includes a number of novel contributions for the multiple-source adaptation problem. We present new normalized solutions with strong theoretical guarantees for the cross-entropy loss and other similar losses. We also provide new…
A pattern of a sequence is a sequence of integer indices with each index describing the order of first occurrence of the respective symbol in the original sequence. In a recent paper, tight general bounds on the block entropy of patterns of…