Related papers: Projection matrices and the sweep operator

In this short note, we establish, for the first time, the convergence rate of SOAP, an efficient and popular matrix-based optimizer for training deep neural networks. Our analysis extends to a more general variant of SOAP that admits…

The undergraduate curriculum in statistics and data science is undergoing changes to accommodate new methods, newly interested students, and the changing role of statistics in society. Because of this, it is more important than ever that…

This paper presents a non-minimal order dynamics model for many analysis, simulation, and control problems of constrained mechanical systems with switching topology by making use of linear projection operator. The distinct features of this…

Random projection is a common technique for designing algorithms in a variety of areas, including information retrieval, compressive sensing and measuring of outlyingness. In this work, the original random projection outlyingness measure is…

Sparseness is a useful regularizer for learning in a wide range of applications, in particular in neural networks. This paper proposes a model targeted at classification tasks, where sparse activity and sparse connectivity are used to…

The gradient scheme framework is based on a small number of properties and encompasses a large number of numerical methods for diffusion models. We recall these properties and develop some new generic tools associated with the gradient…

Experimental designs reflect assumptions about variable relationships that determine what causal queries researchers can answer through the experiment. Accounting for and communicating these assumptions is essential for drawing valid,…

Many online, i.e., time-adaptive, inverse problems in signal processing and machine learning fall under the wide umbrella of the asymptotic minimization of a sequence of non-negative, convex, and continuous functions. To incorporate…

A framework previously introduced in [3] for solving a sequence of stochastic optimization problems with bounded changes in the minimizers is extended and applied to machine learning problems such as regression and classification. The…

This thesis describes work on two applications of probabilistic programming: the learning of probabilistic program code given specifications, in particular program code of one-dimensional samplers; and the facilitation of sequential Monte…

This paper is concerned with the use of the stereographic projection to map the points of a design on the sphere in three dimensions onto a two-dimensional stereogram. Details of the projection and its attendant stereogram are given and the…

The effect of bias on hypothesis formation is characterized for an automated data-driven projection pursuit neural network to extract and select features for binary classification of data streams. This intelligent exploratory process…

It is often desirable to reduce the dimensionality of a large dataset by projecting it onto a low-dimensional subspace. Matrix sketching has emerged as a powerful technique for performing such dimensionality reduction very efficiently. Even…

Operator learning refers to the application of ideas from machine learning to approximate (typically nonlinear) operators mapping between Banach spaces of functions. Such operators often arise from physical models expressed in terms of…

We present a linear algebra formulation of backpropagation which allows the calculation of gradients by using a generically written ``backslash'' or Gaussian elimination on triangular systems of equations. Generally, the matrix elements are…

Most existing generative models are limited to learning a single probability distribution from the training data and cannot generalize to novel distributions for unseen data. An architecture that can generate samples from both trained…

These are expended notes of my talk at the summer institute in algebraic geometry (Seattle, July-August 2005), whose main purpose is to present a global overview on the theory of higher and derived stacks. This text is far from being…

Gradient clipping is widely used to stabilize deep network training, but its formulation as a hard, fixed threshold limits flexibility and ignores gradient distribution dynamics. We propose SPAMP (Statistical Per-layer Adaptive Modulation…

One-dimensional cut-and-project point sets obtained from the square lattice in the plane are considered from a unifying point of view and in the perspective of aperiodic wavelet constructions. We successively examine their geometrical…

Probabilistic forecasting of time series is an important matter in many applications and research fields. In order to draw conclusions from a probabilistic forecast, we must ensure that the model class used to approximate the true…