Generalized Hadamard Product and the Derivatives of Spectral Functions

In this work we propose a generalization of the Hadamard product between two matrices to a tensor-valued, multi-linear product between k matrices for any $k \ge 1$. A multi-linear dual operator to the generalized Hadamard product is presented. It is a natural generalization of the Diag x operator, that maps a vector $x \in \R^n$ into the diagonal matrix with x on its main diagonal. Defining an action of the $n \times n$ orthogonal matrices on the space of k-dimensional tensors, we investigate its interactions with the generalized Hadamard product and its dual. The research is motivated, as illustrated throughout the paper, by the apparent suitability of this language to describe the higher-order derivatives of spectral functions and the tools needed to compute them. For more on the later we refer the reader to [14] and [15], where we use the language and properties developed here to study the higher-order derivatives of spectral functions.