Related papers: Joint Distributions for TensorFlow Probability
Probabilistic programming languages and modeling toolkits are two modular ways to build and reuse stochastic models and inference procedures. Combining strengths of both, we express models and inference as generalized coroutines in the same…
A systematic study of the probability distribution of superimposed random codes is presented through the use of generating functions. Special attention is paid to the cases of either uniformly distributed but not necessarily independent or…
Scaling probabilistic models to large realistic problems and datasets is a key challenge in machine learning. Central to this effort is the development of tractable probabilistic models (TPMs): models whose structure guarantees efficient…
We study machine learning formulations of inductive program synthesis; given input-output examples, we try to synthesize source code that maps inputs to corresponding outputs. Our aims are to develop new machine learning approaches based on…
These are lecture notes for various Summer and Winter schools that I have given. The notes describe the methodology called Variational Modelling, and focus on the application to the modelling of gradient-flow systems. I describe the…
Probabilistic inference provides a language for describing how organisms may learn from and adapt to their environment. The computations needed to implement probabilistic inference often require specific representations, akin to having the…
Dataflow matrix machines are a powerful generalization of recurrent neural networks. They work with multiple types of arbitrary linear streams, multiple types of powerful neurons, and allow to incorporate higher-order constructions. We…
A partial differential equation governing the global evolution of the joint probability distribution of an arbitrary number of local flow observations, drawn randomly from a control volume, is derived and applied to examples involving…
Recursive calls over recursive data are useful for generating probability distributions, and probabilistic programming allows computations over these distributions to be expressed in a modular and intuitive way. Exact inference is also…
We develop a denotational model for probabilistic and concurrent imperative programs, a class of programs with standard control flow via conditionals and while-loops, as well as probabilistic actions and parallel composition. Whereas…
This paper investigates the combinatorics that gives rise to the Boltzmann probability distribution. Despite being one of the most important distributions in physics and other fields of science, the mathematics of the underlying model of…
Dense flow visualization is a popular visualization paradigm. Traditionally, the various models and methods in this area use a continuous formulation, resting upon the solid foundation of functional analysis. In this work, we examine a…
We introduce a new type of graphical model called a "cumulative distribution network" (CDN), which expresses a joint cumulative distribution as a product of local functions. Each local function can be viewed as providing evidence about…
A class of probability distributions is characterized via equalities in law between two order statistics shifted by independent exponential variables. An explicit formula for the quintile function of the identified family of distributions…
A predictive distribution over a sequence of $N+1$ events is said to be "frequency mimicking" whenever the probability for the final event conditioned on the outcome of the first $N$ events equals the relative frequency of successes among…
An expression for the joint probability distribution of the principal curvatures at an arbitrary point in the ensemble of isosurfaces defined on isotropic Gaussian random fields on Rn is derived. The result is obtained by deriving symmetry…
Model ensembling is a technique to combine the predicted distributions of two or more models, often leading to improved robustness and performance. For ensembling in text generation, the next token's probability distribution is derived from…
This book is a graduate-level introduction to probabilistic programming. It not only provides a thorough background for anyone wishing to use a probabilistic programming system, but also introduces the techniques needed to design and build…
The exponential family of models is defined in a general setting, not relying on probability theory. Some results of information geometry are shown to remain valid. Exponential families both of classical and of quantum mechanical…
The embedding space of language models is widely believed to capture the semantic relationships; for instance, embeddings of digits often exhibit an ordered structure that corresponds to their natural sequence. However, the mechanisms…