Related papers: Optimization and Sampling Under Continuous Symmetr…
In this survey, we report on the state of the art of some of the fundamental problems in the Lie theory of Lie groups modeled on locally convex spaces, such as integrability of Lie algebras, integrability of Lie subalgebras to Lie…
Symmetry in differential equations reveals invariances and offers a powerful means to reduce model complexity. Lie group analysis characterizes these symmetries through infinitesimal generators, which provide a local, linear criterion for…
Lie symmetry analysis is one of the powerful tools to analyze nonlinear ordinary differential equations. We review the effectiveness of this method in terms of various symmetries. We present the method of deriving Lie point symmetries,…
Lie theory of continuous transformations provides a unified and powerful approach for handling differential equations. Unfortunately, any small perturbation of an equation usually destroys some important symmetries, and this reduces the…
Nonconvex optimization is central to modern machine learning, but the general framework of nonconvex optimization yields weak convergence guarantees that are too pessimistic compared to practice. On the other hand, while convexity enables…
Optimal values and solutions of empirical approximations of stochastic optimization problems can be viewed as statistical estimators of their true values. From this perspective, it is important to understand the asymptotic behavior of these…
The problem of detecting and quantifying the presence of symmetries in datasets is useful for model selection, generative modeling, and data analysis, amongst others. While existing methods for hard-coding transformations in neural networks…
Large language models (LLMs) are pretrained by minimizing the cross-entropy loss for next-token prediction. In this paper, we study whether this optimization strategy can induce geometric structure in the learned model weights and context…
Beginning with the projectively invariant method for linear programming, interior point methods have led to powerful algorithms for many difficult computing problems, in combinatorial optimization, logic, number theory and non-convex…
Exploiting symmetry inherent in data can significantly improve the sample efficiency of a learning procedure and the generalization of learned models. When data clearly reveals underlying symmetry, leveraging this symmetry can naturally…
Symmetry is the essential element of lifted inference that has recently demon- strated the possibility to perform very efficient inference in highly-connected, but symmetric probabilistic models models. This raises the question, whether…
We consider robust covariance estimation with group symmetry constraints. Non-Gaussian covariance estimation, e.g., Tyler scatter estimator and Multivariate Generalized Gaussian distribution methods, usually involve non-convex minimization…
Dedicated treatment of symmetries in satisfiability problems (SAT) is indispensable for solving various classes of instances arising in practice. However, the exploitation of symmetries usually takes a black box approach. Typically,…
Symmetry principles underlie and guide scientific theory and research, from Curie's invariance formulation to modern applications across physics, chemistry, and mathematics. Building on a recent matrix Lie group measurement model, this…
We propose a general theory for studying the \xl{landscape} of nonconvex \xl{optimization} with underlying symmetric structures \tz{for a class of machine learning problems (e.g., low-rank matrix factorization, phase retrieval, and deep…
Non-convex optimization plays a key role in a growing number of machine learning applications. This motivates the identification of specialized structure that enables sharper theoretical analysis. One such identified structure is…
Symmetry is a powerful tool for finding analytical solutions to differential equations, both partial and ordinary, via the similarity variables or via the invariance of the equation under group transformations. It is the largest group of…
Lie group theory was originally created more than 100 years ago as a tool for solving ordinary and partial differential equations. In this article we review the results of a much more recent program: the use of Lie groups to study…
Statistical models that possess symmetry arise in diverse settings such as random fields associated to geophysical phenomena, exchangeable processes in Bayesian statistics, and cyclostationary processes in engineering. We formalize the…
Symmetry in mathematical programming may lead to a multiplicity of solutions. In nonconvex optimisation, it can negatively affect the performance of the branch-and-bound algorithm. Symmetry may induce large search trees with multiple…