Related papers: Point Divergence Gain and Multidimensional Data Se…
We consider the problem of multi-task learning in the high dimensional setting. In particular, we introduce an estimator and investigate its statistical and computational properties for the problem of multiple connected linear regressions…
The concept of entropy rate for a dynamical process on a graph is introduced. We study diffusion processes where the node degrees are used as a local information by the random walkers. We describe analitically and numerically how the degree…
Recent years have seen a surge of interest in the algorithmic estimation of stochastic entropy production (EP) from trajectory data via machine learning. A crucial element of such algorithms is the identification of a loss function whose…
We review recent developments in detecting and estimating multiple change-points in time series models with exogenous and endogenous regressors, panel data models, and factor models. This review differs from others in multiple ways: (1) it…
There are three ways to conceptualize entropy: entropy as an extensive thermodynamic quantity of physical systems (Clausius, Boltzmann, Gibbs), entropy as a measure for information production of ergodic sources (Shannon), and entropy as a…
We study the Lp-integrated risk of some classical estimators of the density, when the observations are drawn from a strictly stationary sequence. The results apply to a large class of sequences, which can be non-mixing in the sense of…
Entropy estimation is of practical importance in information theory and statistical science. Many existing entropy estimators suffer from fast growing estimation bias with respect to dimensionality, rendering them unsuitable for…
We consider preferential attachment random graphs which may be obtained as follows: It starts with a single node. If a new node appears, it is linked by an edge to one or more existing node(s) with a probability proportional to function of…
Entropy measures of probability distributions are widely used measures in ecology, biology, genetics, and in other fields, to quantify species diversity of a community. Unfortunately, entropy-based diversity indices, or diversity indices…
In the evolving voter model, when an individual interacts with a neighbor having an opinion different from theirs, they will with probability $1-\alpha$ imitate the neighbor but with probability $ \alpha$ will sever the connection and…
We propose a new approach for estimating the parameters of a probability distribution. It consists on combining two new methods of estimation. The first is based on the definition of a new distance measuring the difference between…
This dissertation investigates relative entropies, also called generalized divergences, and how they can be used to characterize information-theoretic tasks in quantum information theory. The main goal is to further refine characterizations…
Statistical properties of coupled dynamic-stochastic systems are studied within a combination of the maximum information principle and the superstatistical approach. The conditions at which the Shannon entropy functional leads to a…
Transition points mark qualitative changes in the macroscopic properties of large complex systems. Explosive transitions, exhibiting properties of both continuous and discontinuous phase transitions, have recently been uncovered in network…
We use phase space distributions specifically, the Wigner distribution (WD) and Husimi distribution (HD) to investigate certain information-theoretic measures as descriptors for a given system. We extensively investigate and analyze…
A generalization of Gauss's principle is used to derive the error laws corresponding to Types II and VII distributions in Pearson's classification scheme. Student's $r$-pdf (Type II) governs the distribution of the internal energy of a…
In several variables, we prove the pointwise convergence of multiresolution expansions to the distributional point values of tempered distributions and distributions of superexponential growth. The article extends and improves earlier…
We develop information-theoretic measures of spatial structure and pattern in more than one dimension. As is well known, the entropy density of a two-dimensional configuration can be efficiently and accurately estimated via a converging…
Estimating information-theoretic quantities such as entropy and mutual information is central to many problems in statistics and machine learning, but challenging in high dimensions. This paper presents estimators of entropy via inference…
Exponential families are widely used in machine learning, including many distributions in continuous and discrete domains (e.g., Gaussian, Dirichlet, Poisson, and categorical distributions via the softmax transformation). Distributions in…