Related papers: Reduced Perplexity: A simplified perspective on as…
Univariate and multivariate normal probability distributions are widely used when modeling decisions under uncertainty. Computing the performance of such models requires integrating these distributions over specific domains, which can vary…
A wide array of machine learning problems are formulated as the minimization of the expectation of a convex loss function on some parameter space. Since the probability distribution of the data of interest is usually unknown, it is is often…
In this paper we propose a method to construct probability measures on the space of convex bodies with a given pushforward distribution. Concretely we show that there is a measure on the metric space of centrally symmetric convex bodies,…
Shannon Entropy is the preeminent tool for measuring the level of uncertainty (and conversely, information content) in a random variable. In the field of communications, entropy can be used to express the information content of given…
Due to instrumental limitations, the nature of which vary from case to case, spectroscopic galaxy redshift surveys usually do not collect redshifts for all galaxies in the population of potential targets. Especially problematic is the…
This report concerns the problem of dimensionality reduction through information geometric methods on statistical manifolds. While there has been considerable work recently presented regarding dimensionality reduction for the purposes of…
Probabilistic representation spaces convey information about a dataset and are shaped by factors such as the training data, network architecture, and loss function. Comparing the information content of such spaces is crucial for…
An approach to build Probabilistic Arithmetic in which initial values of all correlated random variables are known, but with varying degrees of accuracy. As a result of the proposed Probabilistic Arithmetic operations, variable values,…
In the presence of model risk, it is well-established to replace classical expected values by worst-case expectations over all models within a fixed radius from a given reference model. This is the "robustness" approach. We show that…
Probability metrics constitute an important tool in probability theory and statistics \cite{DKS91}, \cite{R91}, \cite{Z83} as they are specific metrics on spaces of random variables which, by satisfying an extra condition, concord well with…
The established language for statistical testing --- significance levels, power, and p-values --- is overly complicated and deceptively conclusive. Even teachers of statistics and scientists who use statistics misinterpret the results of…
This paper introduces a novel extension of Fr\'{e}chet means, called \textit{generalized Fr\'{e}chet means} as a comprehensive framework for characterizing features in probability distributions in general topological spaces. The generalized…
Logistic regression models are a popular and effective method to predict the probability of categorical response data. However inference for these models can become computationally prohibitive for large datasets. Here we adapt ideas from…
This paper introduces an objective metric for evaluating a parsing scheme. It is based on Shannon's original work with letter sequences, which can be extended to part-of-speech tag sequences. It is shown that this regular language is an…
Information-theoretic inequalities play a fundamental role in numerous scientific and technological areas as they generally express the impossibility to have a complete description of a system via a finite number of information measures. In…
Consider a probability measure supported by a regular geodesic ball in a manifold. For any p larger than or equal to 1 we define a stochastic algorithm which converges almost surely to the p-mean of the measure. Assuming furthermore that…
E.T. Jaynes, originator of the maximum entropy interpretation of statistical mechanics, emphasized that there is an inevitable trade-off between the conflicting requirements of robustness and accuracy for any inferencing algorithm. This is…
Perplexity -- a function measuring a model's overall level of "surprise" when encountering a particular output -- has gained significant traction in recent years, both as a loss function and as a simple-to-compute metric of model quality.…
The majority of machine learning methods can be regarded as the minimization of an unavailable risk function. To optimize the latter, given samples provided in a streaming fashion, we define a general stochastic Newton algorithm and its…
A simple method for adding uncertainty to neural network regression tasks via estimation of a general probability distribution is described. The methodology supports estimation of heteroscedastic, asymmetric uncertainties by a simple…