Related papers: Two Lectures on Iterative Coding and Statistical M…
The aim of this review, based on a series of four lectures held at the 22nd "Saalburg" Summer School (2016), is to cover selected topics in the theory of perturbation series and their summation. The first part is devoted to strategies for…
Sparse coding in learned dictionaries has been established as a successful approach for signal denoising, source separation and solving inverse problems in general. A dictionary learning method adapts an initial dictionary to a particular…
We recount recent history behind building compact models of nonlinear, complex processes and identifying their relevant macroscopic patterns or "macrostates". We give a synopsis of computational mechanics, predictive rate-distortion theory,…
Sparse coding and dictionary learning are popular techniques for linear inverse problems such as denoising or inpainting. However in many cases, the measurement process is nonlinear, for example for clipped, quantized or 1-bit measurements.…
Probabilistic Answer Set Programming under the credal semantics (PASP) extends Answer Set Programming with probabilistic facts that represent uncertain information. The probabilistic facts are discrete with Bernoulli distributions. However,…
Graphical modelling techniques based on sparse selection have been applied to infer complex networks in many fields, including biology and medicine, engineering, finance, and social sciences. One structural feature of some of the networks…
Sparse coding is a class of unsupervised methods for learning a sparse representation of the input data in the form of a linear combination of a dictionary and a sparse code. This learning framework has led to state-of-the-art results in…
Flipped classroom pedagogy is widely used in undergraduate mathematics to promote active learning, yet it remains unclear whether students experience it in systematically different ways. In this study, we analyze student perceptions from an…
We are convinced of the usefulness of sketches and diagrams during mathematical work but the observation is made in our practices that they are not spontaneously used by students. In order to study the understanding and use of sketches by…
Sparse graphical modelling has attained widespread attention across various academic fields. We propose two new graphical model approaches, Gslope and Tslope, which provide sparse estimates of the precision matrix by penalizing its sorted…
We study the theoretical performance of a combined approach to demodulation and decoding of binary continuous-phase modulated signals under repetition-like codes. This technique is motivated by a need to transmit packetized or framed data…
The number partitioning problem consists of partitioning a sequence of positive numbers ${a_1,a_2,..., a_N}$ into two disjoint sets, ${\cal A}$ and ${\cal B}$, such that the absolute value of the difference of the sums of $a_j$ over the two…
This is a set of lecture notes that developed out of courses on the lambda calculus that I taught at the University of Ottawa in 2001 and at Dalhousie University in 2007 and 2013. Topics covered in these notes include the untyped lambda…
We study a new class of codes for Gaussian multi-terminal source and channel coding. These codes are designed using the statistical framework of high-dimensional linear regression and are called Sparse Superposition or Sparse Regression…
Neural networks and other machine learning models compute continuous representations, while humans communicate with discrete symbols. Reconciling these two forms of communication is desirable to generate human-readable interpretations or to…
The performance of an error correcting code is evaluated by its error probability, rate, and en/decoding complexity. The performance of a series of codes is evaluated by, as the block lengths approach infinity, whether their error…
The treatment of the number-theoretical problem of integer partitions within the approach of statistical mechanics is discussed. Historical overview is given and known asymptotic results for linear and plane partitions are reproduced. From…
When dealing with datasets containing a billion instances or with simulations that require a supercomputer to execute, computational resources become part of the equation. We can improve the efficiency of learning and inference by…
In "On Coding for Reliable Communication over Packet Networks" (Lun, Medard, and Effros, Proc. 42nd Annu. Allerton Conf. Communication, Control, and Computing, 2004), a capacity-achieving coding scheme for unicast or multicast over lossy…
Probabilistic programming is the idea of writing models from statistics and machine learning using program notations and reasoning about these models using generic inference engines. Recently its combination with deep learning has been…