Related papers: Prove Costa's Entropy Power Inequality and High Or…
This paper deals with uncertainty quantification and out-of-distribution detection in deep learning using Bayesian and ensemble methods. It proposes a practical solution to the lack of prediction diversity observed recently for standard…
We show that a proper expression of the uncertainty relation for a pair of canonically-conjugate continuous variables relies on entropy power, a standard notion in Shannon information theory for real-valued signals. The resulting…
The Expectation-Maximization (EM) algorithm for mixture models often results in slow or invalid convergence. The popular convergence proof affirms that the likelihood increases with Q; Q is increasing in the M -step and non-decreasing in…
This paper extends the high-order entropy stable (ES) adaptive moving mesh finite difference schemes developed in [14] to the two- and three-dimensional (multi-component) compressible Euler equations with the stiffened equation of state.…
The combinatorial basis of entropy by Boltzmann can be written $H= {N}^{-1} \ln \mathbb{W}$, where $H$ is the dimensionless entropy of a system, per unit entity, $N$ is the number of entities and $\mathbb{W}$ is the number of ways in which…
The Contrastive Divergence (CD) algorithm has achieved notable success in training energy-based models including Restricted Boltzmann Machines and played a key role in the emergence of deep learning. The idea of this algorithm is to…
Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension…
We study entropy-bounded computational geometry, that is, geometric algorithms whose running times depend on a given measure of the input entropy. Specifically, we introduce a measure that we call range-partition entropy, which unifies and…
Classical and quantum physics impose different constraints on the joint probability distributions of observed variables in a causal structure. These differences mean that certain correlations can be certified as non-classical, which has…
Finding the minimal relative entropy of two quantum states under semidefinite constraints is a pivotal problem located at the mathematical core of various applications in quantum information theory. An efficient method for providing…
We revisit the well-studied problem of estimating the Shannon entropy of a probability distribution, now given access to a probability-revealing conditional sampling oracle. In this model, the oracle takes as input the representation of a…
In quantum information theory, entropic inequalities act as the necessary and sufficient conditions to noncontextuality. Here, we first experimentally demonstrate the violation of the entropic noncontextual inequality in a four-level…
Solutions to conservation laws satisfy the monotonicity property: the number of local extrema is a non-increasing function of time, and local maximum/minimum values decrease/increase monotonically in time. This paper investigates this…
The quantum entropy power inequality, proven by K\"onig and Smith (2012), states that $\exp(S(\rho \boxplus \sigma)/m)\geq \frac 12 (\exp(S(\rho)/m) + \exp(S(\sigma)/m))$ for two $m$-mode bosonic quantum states $\rho$ and $\sigma$. One…
In this paper we consider splitting methods for nonlinear ordinary differential equations in which one of the (partial) flows that results from the splitting procedure can not be computed exactly. Instead, we insert a well-chosen state…
By using the $\Phi$-entropy inequality derived in \cite{Wu, Ch} for Poisson measures, the same type of inequality is established for a class of stochastic differential equations driven by purely jump L\'evy processes. The semigroup…
Simulating an arbitrary discrete distribution $D \in [0, 1]^n$ using fair coin tosses incurs trade-offs between entropy complexity and space and time complexity. Shannon's theory suggests that $H(D)$ tosses are necessary and sufficient, but…
High order schemes are known to be unstable in the presence of shock discontinuities or under-resolved solution features for nonlinear conservation laws. Entropy stable schemes address this instability by ensuring that physically relevant…
We consider the real $\beta$-ensemble (or 1D log-gas) of dimension $N$ in the high-temperature regime, \textit{i.e.} where the inverse temperature $\beta$ scales as $N\beta=2P$ with $P$ a fixed positive parameter. We establish the large-$N$…
In this article we provide initial findings regarding the problem of solving likelihood equations by means of a maximum entropy approach. Unlike standard procedures that require equating at zero the score function of the maximum-likelihood…