Related papers: Sparse extractor families for all the entropy
We study minimization of a parametric family of relative entropies, termed relative $\alpha$-entropies (denoted $\mathscr{I}_{\alpha}(P,Q)$). These arise as redundancies under mismatched compression when cumulants of compressed lengths are…
Estimating entropies from limited data series is known to be a non-trivial task. Naive estimations are plagued with both systematic (bias) and statistical errors. Here, we present a new 'balanced estimator' for entropy functionals Shannon,…
Recently there has been an increasing interest in methods that deal with multiple outputs. This has been motivated partly by frameworks like multitask learning, multisensor networks or structured output data. From a Gaussian processes…
Several implications of well-known fluctuation theorems, on the statistical properties of the entropy production, are studied using various approaches. We begin by deriving a tight lower bound on the variance of the entropy production for a…
Sparse tensor networks are commonly used to represent contractions over sparse tensors. Tensor contractions are higher-order analogs of matrix multiplication. Tensor networks arise commonly in many domains of scientific computing and data…
Compressive sensing has been successfully used for optimized operations in wireless sensor networks. However, raw data collected by sensors may be neither originally sparse nor easily transformed into a sparse data representation. This…
In this paper, we propose new semiparametric procedures for making inference on linear functionals and their functions of two semicontinuous populations. The distribution of each population is usually characterized by a mixture of a…
Given a general scalar balance law, i.e., in several space dimensions and with flux and source both space and time dependent, we focus on the functional properties of the entropy production. We apply this operator to entropy solutions, to…
We introduce an interesting method of proving separable reduction theorems - the method of elementary submodels. We are studying whether it is true that a set (function) has given property if and only if it has this property with respect to…
Maximum-entropy ensembles are key primitives in statistical mechanics from which thermodynamic properties can be derived. Over the decades, several approaches have been put forward in order to justify from minimal assumptions the use of…
Sparsity is a desirable attribute. It can lead to more efficient and more effective representations compared to the dense model. Meanwhile, learning sparse latent representations has been a challenging problem in the field of computer…
A random variable $X$ is an $(n,k)$-zero-fixing source if for some subset $V\subseteq[n]$, $X$ is the uniform distribution on the strings $\{0,1\}^n$ that are zero on every coordinate outside of $V$. An $\epsilon$-extractor for…
The Cross Entropy method is a well-known adaptive importance sampling method for rare-event probability estimation, which requires estimating an optimal importance sampling density within a parametric class. In this article we estimate an…
We propose an entropy function for simplicial complices. Its value gives the expected cost of the optimal encoding of sequences of vertices of the complex, when any two vertices belonging to the same simplex are indistinguishable. We show…
We consider sampling in the so-called low-temperature regime, which is typically characterised by non-local behaviour and strong global correlations. Canonical examples include sampling independent sets on bipartite graphs and sampling from…
Standard sparse pseudo-input approximations to the Gaussian process (GP) cannot handle complex functions well. Sparse spectrum alternatives attempt to answer this but are known to over-fit. We suggest the use of variational inference for…
We study beyond worst-case dimensionality reduction for $s$-sparse vectors. Our work is divided into two parts, each focusing on a different facet of beyond worst-case analysis: We first consider average-case guarantees. A folklore upper…
A fundamental concept in control theory is that of controllability, where any system state can be reached through an appropriate choice of control inputs. Indeed, a large body of classical and modern approaches are designed for controllable…
We introduce Tiered Sampling, a novel technique for approximate counting sparse motifs in massive graphs whose edges are observed in a stream. Our technique requires only a single pass on the data and uses a memory of fixed size $M$, which…
This paper shows how to generate code that efficiently converts sparse tensors between disparate storage formats (data layouts) such as CSR, DIA, ELL, and many others. We decompose sparse tensor conversion into three logical phases:…