Related papers: The Entropy of Backwards Analysis
An iterative randomness extraction algorithm which generalized the Von Neumann's extraction algorithm is detailed, analyzed and implemented in standard C++. Given a sequence of independently and identically distributed biased Bernoulli…
The purpose of model selection algorithms such as All Subsets, Forward Selection and Backward Elimination is to choose a linear model on the basis of the same set of data to which the model will be applied. Typically we have available a…
The minimum average number of bits need to describe a random variable is its entropy, assuming knowledge of the underlying statistics On the other hand, universal compression supposes that the distribution of the random variable, while…
This paper considers properties of an optimization based sampler for targeting the posterior distribution when the likelihood is intractable and auxiliary statistics are used to summarize information in the data. Our reverse sampler…
Entropy rate is a real valued functional on the space of discrete random sources which lacks a closed formula even for subclasses of sources which have intuitive parameterizations. A good way to overcome this problem is to examine its…
Graded posets frequently arise throughout combinatorics, where it is natural to try to count the number of elements of a fixed rank. These counting problems are often $\#\textbf{P}$-complete, so we consider approximation algorithms for…
Generating random bits from a source of biased coins (the biased is unknown) is a classical question that was originally studied by von Neumann. There are a number of known algorithms that have asymptotically optimal information efficiency,…
Problems of probabilistic inference and decision making under uncertainty commonly involve continuous random variables. Often these are discretized to a few points, to simplify assessments and computations. An alternative approximation is…
Low Rank Approximation is among most fundamental subjects of numerical linear algebra having important applications to various areas of modern computing and %they range from machine learning theory and %neural networks to data mining and…
Let $\{A_{i,n}\}$ be a triangular array of elements in a Banach algebra, whose norms do not grow too fast, and whose row averages converge to $A$. Let $\sigma \in S(n)$ be a permutation drawn uniformly at random. If the array only contains…
Given a set of observations generated by an optimization process, the goal of inverse optimization is to determine likely parameters of that process. We cast inverse optimization as a form of deep learning. Our method, called deep inverse…
Given a stream of Bernoulli random variables, consider the problem of estimating the mean of the random variable within a specified relative error with a specified probability of failure. Until now, the Gamma Bernoulli Approximation Scheme…
Finite sample properties of random covariance-type matrices have been the subject of much research. In this paper we focus on the "lower tail" of such a matrix, and prove that it is subgaussian under a simple fourth moment assumption on the…
In many applications, Bayesian inverse problems can give rise to probability distributions which contain complexities due to the Hessian varying greatly across parameter space. This complexity often manifests itself as lower dimensional…
Bounds on the entropy of patterns of sequences generated by independently identically distributed (i.i.d.) sources are derived. A pattern is a sequence of indices that contains all consecutive integer indices in increasing order of first…
A novel heuristic approach is proposed here for time series data analysis, dubbed Generalized weighted permutation entropy, which amalgamates and generalizes beyond their original scope two well established data analysis methods:…
We study a twist on the classic secretary problem, which we term the secretary ranking problem: elements from an ordered set arrive in random order and instead of picking the maximum element, the algorithm is asked to assign a rank, or…
In this paper, we consider robust control using randomized algorithms. We extend the existing order statistics distribution theory to the general case in which the distribution of population is not assumed to be continuous and the order…
The relative entropy is a principal measure of distinguishability in quantum information theory, with its most important property being that it is non-increasing with respect to noisy quantum operations. Here, we establish a remainder term…
Nonlinear dynamics play an important role in the analysis of signals. A popular, readily interpretable nonlinear measure is Permutation Entropy. It has recently been extended for the analysis of graph signals, thus providing a framework for…