相关论文: Extractors: Low Entropy Requirements Colliding Wit…
This paper considers the problem of lossy compression for the computation of a function of two correlated sources, both of which are observed at the encoder. Due to presence of observation costs, the encoder is allowed to observe only…
We study channel resolvability for the discrete memoryless multiple-access channel with cribbing, i.e., the characterization of the amount of randomness required at the inputs to approximately produce a chosen i.i.d. output distribution…
The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible constrained to match empirical data, for instance, feature expectations. We seek to generalize…
This paper investigates achievable information rates and error exponents of mismatched decoding when the channel belongs to the class of channels that are close to the decoding metric in terms of relative entropy. For both discrete- and…
There are currently two main, continuum models of entropy: a 'reversible', Clausius entropy model and an 'irreversible', Onsager-Prigogine entropy model. It is shown that the equations of the 'reversible' and the 'irreversible' entropy…
(Abridged) We present a new method for detecting and measuring compact sources in conditions of intense, and highly variable, fore/background. While all most commonly used packages carry out the source detection over the signal image, our…
This paper addresses the critical challenge of estimating the reliability of an Electric Vehicle (EV) charging systems when facing risks such as overheating, unpredictable, weather, and cyberattacks. Traditional methods for predicting…
We introduce an online outlier detection algorithm to detect outliers in a sequentially observed data stream. For this purpose, we use a two-stage filtering and hedging approach. In the first stage, we construct a multi-modal probability…
Distribution matching is the process of invertibly mapping a uniformly distributed input sequence onto sequences that approximate the output of a desired discrete memoryless source. The special case of a binary output alphabet and…
Information theory provides principled ways to analyze different inference and learning problems such as hypothesis testing, clustering, dimensionality reduction, classification, among others. However, the use of information theoretic…
A recent experiment showed that, contrary to theoretical predictions, beyond a cutoff point grinding coffee more finely results in lower extraction. One potential explanation for this is that fine grinding promotes non-uniform extraction in…
Large datasets are often affected by cell-wise outliers in the form of missing or erroneous data. However, discarding any samples containing outliers may result in a dataset that is too small to accurately estimate the covariance matrix.…
In the present work we study the well-known Two Capacitor Problem from a new perspective. Although this problem has been thoroughly investigated, as far as we know there are no studies of the thermodynamic aspects of the discharge process.…
Scalable sampling of molecular states in thermodynamic equilibrium is a long-standing challenge in statistical physics. Boltzmann generators tackle this problem by pairing normalizing flows with importance sampling to obtain uncorrelated…
We study the problem of entropy calibration, which asks whether a language model's entropy over generations matches its log loss on human text. Past work found that models are miscalibrated, with entropy per step increasing as generations…
We present a detailed derivation of some estimators of Shannon entropy for discrete distributions. They hold for finite samples of N points distributed into M "boxes", with N and M -> oo, but N/M < oo. In the high sampling regime (<< 1…
Tensors play a central role in many modern machine learning and signal processing applications. In such applications, the target tensor is usually of low rank, i.e., can be expressed as a sum of a small number of rank one tensors. This…
We discuss entropy production in nonequilibrium steady states by focusing on paths obtained by sampling at regular (small) intervals, instead of sampling on each change of the system's state. This allows us to study directly entropy…
The combinatorial basis of entropy by Boltzmann can be written $H= {N}^{-1} \ln \mathbb{W}$, where $H$ is the dimensionless entropy of a system, per unit entity, $N$ is the number of entities and $\mathbb{W}$ is the number of ways in which…
Consider the problem of estimating the Shannon entropy of a distribution over $k$ elements from $n$ independent samples. We show that the minimax mean-square error is within universal multiplicative constant factors of $$\Big(\frac{k }{n…