Related papers: Orthogonal Invariance and Identifiability
Over the past decades, the concept "partial smoothness" has been playing as a powerful tool in several fields involving nonsmooth analysis, such as nonsmooth optimization, inverse problems and operation research, etc. The essence of partial…
Identifiability means that iterates generated by optimization algorithms are eventually confined to an identifiable set. This property is computationally useful because minimizing a nonsmooth function near a critical point reduces to…
Diverse optimization algorithms correctly identify, in finite time, intrinsic constraints that must be active at optimality. Analogous behavior extends beyond optimization to systems involving partly smooth operators, and in particular to…
One of basic difficulties of machine learning is handling unknown rotations of objects, for example in image recognition. A related problem is evaluation of similarity of shapes, for example of two chemical molecules, for which direct…
Identifiability, and the closely related idea of partial smoothness, unify classical active set methods and more general notions of solution structure. Diverse optimization algorithms generate iterates in discrete time that are eventually…
Numerical solutions of differential equations are usually not smooth functions. However, they should resemble the smoothness of the corresponding real solutions in one way or another. In two of our recent papers, a kind of spacial…
Symmetry -- invariance to certain operators -- is a fundamental concept in many branches of physics. We propose ways to measure symmetric properties of vertices, and their surroundings, in networks. To be stable to the randomness inherent…
In this article, we investigate how the entrywise application of a non-linear function to symmetric orthogonally invariant random matrix ensembles alters the spectral distribution. We treat also the multivariate case where we apply…
In this paper, our focus lies on the study of the second-order variational analysis of orthogonally invariant matrix functions. It is well-known that an orthogonally invariant matrix function is an extended-real-value function defined on…
The maximum (or minimum) generalized eigenvalue of symmetric positive semidefinite matrices that depend on optimization variables often appears as objective or constraint functions in structural topology optimization when we consider…
Orbifold equivalence is a notion of symmetry that does not rely on group actions. Among other applications, it leads to surprising connections between hitherto unrelated singularities. While the concept can be defined in a very general…
Critical points of an invariant function may or may not be symmetric. We prove, however, that if a symmetric critical point exists, those adjacent to it are generically symmetry breaking. This mathematical mechanism is shown to carry…
A large number matrix optimization problems are described by orthogonally invariant norms. This paper is devoted to the study of variational analysis of the orthogonally invariant norm cone of symmetric matrices. For a general orthogonally…
A class of parametric functions formed by alternating compositions of multivariate polynomials and rectification style monomial maps is studied (the layer-wise exponents are treated as fixed hyperparameters and are not optimized). For this…
A general invariant manifold theorem is needed to study the topological classes of smooth dynamical systems. These classes are often invariant under renormalization. The classical invariant manifold theorem cannot be applied, because the…
Symmetry plays a crucial role in understanding the properties of mathematical structures and optimization problems. Recent work has explored this phenomenon in the context of neural networks, where the loss function is invariant under…
Many contemporary applications in signal processing and machine learning give rise to structured non-convex non-smooth optimization problems that can often be tackled by simple iterative methods quite effectively. One of the keys to…
We study convolutional neural networks with monomial activation functions. Specifically, we prove that their parameterization map is regular and is an isomorphism almost everywhere, up to rescaling the filters. By leveraging on tools from…
The Eigendecomposition of quadratic forms (symmetric matrices) guaranteed by the spectral theorem is a foundational result in applied mathematics. Motivated by a shared structure found in inferential problems of recent interest---namely…
Model reparametrization, which follows the change-of-variable rule of calculus, is a popular way to improve the training of neural nets. But it can also be problematic since it can induce inconsistencies in, e.g., Hessian-based flatness…