Related papers: Optimum Self Random Number Generation Rate and Its…
A recent line of ground-breaking results for permutation-based SGD has corroborated a widely observed phenomenon: random permutations offer faster convergence than with-replacement sampling. However, is random optimal? We show that this…
In this paper, we revisit the design of Raptor codes for binary input additive white Gaussian noise (BIAWGN) channels, where we are interested in very low signal to noise ratios (SNRs). A linear programming degree distribution optimization…
Statistical divergences (SDs), which quantify the dissimilarity between probability distributions, are a basic constituent of statistical inference and machine learning. A modern method for estimating those divergences relies on…
This paper deals with the computation of a non-asymptotic lower bound by means of the nonanticipative rate-distortion function (NRDF) on the discrete-time zero-delay variable-rate lossy compression problem for discrete Markov sources with…
It is well known that separation between lossy source coding and channel coding is asymptotically optimal under classical additive distortion measures. Recently, coding under a new class of quality considerations, often referred to as…
The joint nonanticipative rate distortion function (NRDF) for a tuple of random processes with individual fidelity criteria is considered. Structural properties of optimal test channel distributions are derived. Further, for the application…
For many applications in signal processing and machine learning, we are tasked with minimizing a large sum of convex functions subject to a large number of convex constraints. In this paper, we devise a new random projection method (RPM) to…
In this work, a method of random parameters generation for randomized learning of a single-hidden-layer feedforward neural network is proposed. The method firstly, randomly selects the slope angles of the hidden neurons activation functions…
We study the fundamental problem of estimating an unknown discrete distribution $p$ over $d$ symbols, given $n$ i.i.d. samples from the distribution. We are interested in minimizing the KL divergence between the true distribution and the…
For the HB problem with the CR constraint, the rate-distortion function is derived under the assumption that the side information sequences are (stochastically) degraded. The rate-distortion function is also calculated explicitly for three…
While SGD, which samples from the data with replacement is widely studied in theory, a variant called Random Reshuffling (RR) is more common in practice. RR iterates through random permutations of the dataset and has been shown to converge…
Neural networks are usually trained by some form of stochastic gradient descent (SGD)). A number of strategies are in common use intended to improve SGD optimization, such as learning rate schedules, momentum, and batching. These are…
This paper studies fixed-rate randomized vector quantization under the constraint that the quantizer's output has a given fixed probability distribution. A general representation of randomized quantizers that includes the common models in…
This work presents two significant contributions from the perspectives of quantum random number generator (QRNG) manufacturers and users. For manufacturers, the conventional method of assessing the quantumness of single-photon-based QRNGs…
In this paper, we approximate the hidden Markov model of chaotic-map truly random number generators (TRNGs) and describe its fundamental limits based on the approximate entropy-rate of the underlying bit-generation process. We demonstrate…
The idea of adversarial learning of regularization functionals has recently been introduced in the wider context of inverse problems. The intuition behind this method is the realization that it is not only necessary to learn the basic…
We propose a rejection sampling scheme using the discriminator of a GAN to approximately correct errors in the GAN generator distribution. We show that under quite strict assumptions, this will allow us to recover the data distribution…
In this paper, we consider a pseudo-random generator based on the difficulty of the syndrome decoding problem for rank metric codes. We also study the resistance of this problem against a quantum computer. Our results show that with rank…
Consider the problem where a statistician in a two-node system receives rate-limited information from a transmitter about marginal observations of a memoryless process generated from two possible distributions. Using its own observations,…
We consider solving a convex, possibly stochastic optimization problem over a randomly time-varying multi-agent network. Each agent has access to some local objective function, and it only has unbiased estimates of the gradients of the…