Related papers: Matrix Calculus (for Machine Learning and Beyond)
We consider the differentiation of the value function for parametric optimization problems. Such problems are ubiquitous in Machine Learning applications such as structured support vector machines, matrix factorization and min-min or…
Linear systems are the bedrock of virtually all numerical computation. Machine learning poses specific challenges for the solution of such systems due to their scale, characteristic structure, stochasticity and the central role of…
We present a methodology for parallel acceleration of learning in the presence of matrix orthogonality and unitarity constraints of interest in several branches of machine learning. We show how an apparently sequential elementary rotation…
Finite differences have been widely used in mathematical theory as well as in scientific and engineering computations. These concepts are constantly mentioned in calculus. Most frequently-used difference formulas provide excellent…
Can a machine learn Machine Learning? This work trains a machine learning model to solve machine learning problems from a University undergraduate level course. We generate a new training set of questions and answers consisting of course…
Machine learning assumes a pivotal role in our data-driven world. The increasing scale of models and datasets necessitates quick and reliable algorithms for model training. This dissertation investigates adaptivity in machine learning…
Matrix regression plays an important role in modern data analysis due to its ability to handle complex relationships involving both matrix and vector variables. We propose a class of regularized regression models capable of predicting both…
Many numerical methods for evaluating matrix functions can be naturally viewed as computational graphs. Rephrasing these methods as directed acyclic graphs (DAGs) is a particularly effective approach to study existing techniques, improve…
In this paper, we introduce the new construction of fractional derivatives and integrals with respect to a function, based on a matrix approach. We believe that this is a powerful tool in both analytical and numerical calculations. We begin…
This is an introductory machine-learning course specifically developed with STEM students in mind. Our goal is to provide the interested reader with the basics to employ machine learning in their own projects and to familiarize themself…
This paper is concerned with the efficient evaluation of higher-order derivatives of functions $f$ that are composed of matrix operations. I.e., we want to compute the $D$-th derivative tensor $\nabla^D f(X) \in \mathbb R^{N^D}$, where…
A new scaling and recovering algorithm is proposed for simultaneously computing the matrix $\varphi$-functions that arise in exponential integrator methods for the numerical solution of certain first-order systems of ordinary differential…
In [1, 2], we have explored the theoretical aspects of feature extraction optimization processes for solving largescale problems and overcoming machine learning limitations. Majority of optimization algorithms that have been introduced in…
This text is a support for different courses of the master of Mechanics of the University Paris-Saclay. The content of this text is an introduction, for graduate students, to tensor algebra and analysis. Far from being exhaustive, the text…
Tuning scientific and probabilistic machine learning models $-$ for example, partial differential equations, Gaussian processes, or Bayesian neural networks $-$ often relies on evaluating functions of matrices whose size grows with the data…
The paper concerns the second-order generalized differentiation theory of variational analysis and new applications of this theory to some problems of constrained optimization in finitedimensional spaces. The main attention is paid to the…
CUR and low-rank approximations are among most fundamental subjects of numerical linear algebra, with a wide range of applications to a variety of highly important areas of modern computing, which range from the machine learning theory and…
This book provides a comprehensive and accessible introduction to the emerging field of AI for mathematics. It covers the core principles and diverse applications of using artificial intelligence to advance mathematical research. Through…
These notes focus on the minimization of convex functionals using first-order optimization methods, which are fundamental in many areas of applied mathematics and engineering. The primary goal of this document is to introduce and analyze…
A characteristic feature of differential-algebraic equations is that one needs to find derivatives of some of their equations with respect to time, as part of so called index reduction or regularisation, to prepare them for numerical…