Related papers: Generalized Probability Smoothing
We study maximum likelihood estimation for spatial generalized linear mixed models with Gaussian process approximations using a stochastic Newton-Raphson algorithm. We consider two Gaussian Process approximations in this context: spectral…
Long samples of text from neural language models can be of poor quality. Truncation sampling algorithms--like top-$p$ or top-$k$ -- address this by setting some words' probabilities to zero at each step. This work provides framing for the…
We study the redundancy of universally compressing strings $X_1,\dots, X_n$ generated by a binary Markov source $p$ without any bound on the memory. To better understand the connection between compression and estimation in the Markov…
Probability metrics have become an indispensable part of modern statistics and machine learning, and they play a quintessential role in various applications, including statistical hypothesis testing and generative modeling. However, in a…
Suppose that an infinite lattice gas of constant density $n_0$, whose dynamics are described by the symmetric simple exclusion process, is brought in contact with a spherical absorber of radius $R$. Employing the macroscopic fluctuation…
The idea of slicing divergences has been proven to be successful when comparing two probability measures in various machine learning applications including generative modeling, and consists in computing the expected value of a `base…
This paper presents new lower and upper bounds for the compression rate of binary prefix codes optimized over memoryless sources according to various nonlinear codeword length objectives. Like the most well-known redundancy bounds for…
For a sample of Exponentially distributed durations we aim at point estimation and a confidence interval for its parameter. A duration is only observed if it has ended within a certain time interval, determined by a Uniform distribution.…
The trapping redundancy of a linear code is the number of rows of a smallest parity-check matrix such that no submatrix forms an $(a,b)$-trapping set. This concept was first introduced in the context of low-density parity-check (LDPC) codes…
We study the generalization error of randomized learning algorithms -- focusing on stochastic gradient descent (SGD) -- using a novel combination of PAC-Bayes and algorithmic stability. Importantly, our generalization bounds hold for all…
The accuracy of compound Poisson approximation to the sum $S=w_1S_1+w_2S_2+...+w_NS_N$ is estimated. Here $S_i$ are sums of independent or weakly dependent random variables, and $w_i$ denote weights. The overall smoothing effect of $S$ on…
We probabilistically analyze the performance of the arithmetic coding algorithm under a probability model for binary data in which a message is received by a coder from a source emitting independent equally distributed bits, with 1…
Consider a probability measure supported by a regular geodesic ball in a manifold. For any p larger than or equal to 1 we define a stochastic algorithm which converges almost surely to the p-mean of the measure. Assuming furthermore that…
Neural autoregressive sequence models smear the probability among many possible sequences including degenerate ones, such as empty or repetitive sequences. In this work, we tackle one specific case where the model assigns a high probability…
This announcement describes a probabilistic approach to cascades which, in addition to providing an entirely probabilistic proof of the Kahane-Peyri\`ere theorem for independent cascades, readily applies to general dependent cascades.…
The numerical accuracy of particle-based approximations in Smoothed Particle Hydrodynamics (SPH) is significantly affected by the spatial uniformity of particle distributions, especially for second-order derivatives. This study aims to…
This paper develops a unified and computationally efficient method for change-point estimation along the time dimension in a non-stationary spatio-temporal process. By modeling a non-stationary spatio-temporal process as a piecewise…
We investigate probabilistic transformations of quantum states from a `source' set to a `target' set of states. Such transforms have many applications. They can be used for tasks which include state-dependent cloning or quantum state…
Given a collection of strings, each with an associated probability of occurrence, the guesswork of each of them is their position in a list ordered from most likely to least likely, breaking ties arbitrarily. Guesswork is central to several…
We propose Partition Tree, a novel tree-based framework for conditional density estimation over general outcome spaces that supports both continuous and categorical variables within a unified formulation. Our approach models conditional…