Related papers: Matrix Calculus (for Machine Learning and Beyond)
For a real function, automatic differentiation is such a standard algorithm used to efficiently compute its gradient, that it is integrated in various neural network frameworks. However, despite the recent advances in using complex…
Automatic differentiation is everywhere, but there exists only minimal documentation of how it works in complex arithmetic beyond stating "derivatives in $\mathbb{C}^d$" $\cong$ "derivatives in $\mathbb{R}^{2d}$" and, at best, shallow…
The paper is devoted to developing subdifferential theory for set-valued mappings taking values in ordered infinite-dimensional spaces. This study is motivated by applications to problems of vector and set optimization with various…
This paper investigates the use of artificial neural networks (ANNs) to solve differential equations (DEs) and the construction of the loss function which meets both differential equation and its initial/boundary condition of a certain DE.…
We propose a machine learning framework to accelerate numerical computations of time-dependent ODEs and PDEs. Our method is based on recasting (generalizations of) existing numerical methods as artificial neural networks, with a set of…
Recent advancements in the realm of deep learning, particularly in the development of large language models (LLMs), have demonstrated AI's ability to tackle complex mathematical problems or solving programming challenges. However, the…
A matrix approach to continuous iteration is proposed for general formal series. It leads, in particular, to an order{to{order iteration of the exponential function, and consequently to an algorithmic approach to tetration. Lower{order…
This overview article highlights the critical role of mathematics in artificial intelligence (AI), emphasizing that mathematics provides tools to better understand and enhance AI systems. Conversely, AI raises new problems and drives the…
Machine learning and neural network models in particular have been improving the state of the art performance on many artificial intelligence related tasks. Neural network models are typically implemented using frameworks that perform…
Expressions for the derivatives with respect to order of modified Bessel functions evaluated at integer orders and certain integral representations of associated Legendre functions with modulus argument greater than unity are used to…
Covariance matrix reconstruction has been the most widely used guiding objective in gridless direction-of-arrival (DoA) estimation for sparse linear arrays. Many semidefinite programming (SDP)-based methods fall under this category.…
Automatic differentiation (AD) is a technique for computing the derivative of a function represented by a program. This technique is considered as the de-facto standard for computing the differentiation in many machine learning and…
We create a sequence version of calculus. First, we define equivalence, some fundamental operations, differential, and integral for sequences. Then, we propose sequence versions of identity function, power function, exponential function,…
This article serves as the regression analysis lecture notes in the Intelligent Computing course cluster (including the courses of Artificial Intelligence, Data Mining, Machine Learning, and Pattern Recognition). It aims to provide students…
This work explores fundamental modeling and algorithmic issues arising in the well-established MapReduce framework. First, we formally specify a computational model for MapReduce which captures the functional flavor of the paradigm by…
Scaling problems have a rich and diverse history, and thereby have found numerous applications in several fields of science and engineering. For instance, the matrix scaling problem has had applications ranging from theoretical computer…
Matrix completion has received vast amount of attention and research due to its wide applications in various study fields. Existing methods of matrix completion consider only nonlinear (or linear) relations among entries in a data matrix…
An analytical-numeric calculation method of extremely complicated integrals is presented. These integrals appear often in magnet soliton theory. The appropriate analytical continuation and a corresponding integration contour allow to reduce…
Automatic differentiation (AD) has been a topic of interest for researchers in many disciplines, with increased popularity since its application to machine learning and neural networks. Although many researchers appreciate and know how to…
Algorithmic differentiation (AD) allows exact computation of derivatives given only an implementation of an objective function. Although many AD tools are available, a proper and efficient implementation of AD methods is not…