Twice Epi-Differentiability of Orthogonally Invariant Matrix Functions and Application

In this paper, our focus lies on the study of the second-order variational analysis of orthogonally invariant matrix functions. It is well-known that an orthogonally invariant matrix function is an extended-real-value function defined on ${\mathbb M}_{m,n}\,(n \leqslant m)$ of the form $f \circ \sigma$ for an absolutely symmetric function $f \colon \R^n \rightarrow [-\infty,+\infty]$ and the singular values $\sigma \colon {\mathbb M}_{m,n} \rightarrow \R^{n}$. We establish several second-order properties of orthogonally invariant matrix functions, such as parabolic epi-differentiability, parabolic regularity, and twice epi-differentiability when their associated absolutely symmetric functions enjoy some properties. Specifically, we show that the nuclear norm of a real $m \times n$ matrix is twice epi-differentiable and we derive an explicit expression of its second-order epi-derivative. Moreover, for a convex orthogonally invariant matrix function, we calculate its second subderivative and present sufficient conditions for twice epi-differentiability. This enables us to establish second-order optimality conditions for a class of matrix optimization problems.