Related papers: On subdifferential chain rule of matrix factorizat…
Large language models (LLMs) have shown their power in different areas. Attention computation, as an important subroutine of LLMs, has also attracted interests in theory. Recently the static computation and dynamic maintenance of attention…
We study an abstract second order inclusion involving two nonlinear single-valued operators and a nonlinear multivalued term. Our goal is to establish the existence of solutions to the problem by applying numerical scheme based on time…
Examples exist of extended-real-valued closed functions on ${\bf R}^n$ whose subdifferentials (in the standard, limiting sense) have large graphs. By contrast, if such a function is semi-algebraic, then its subdifferential graph must have…
Matrix Factorization has emerged as a widely adopted framework for modeling data exhibiting low-rank structures. To address challenges in manifold learning, this paper presents a subspace-constrained quadratic matrix factorization model.…
The paper deals with a comprehensive theory of mappings, whose local behavior can be described by means of linear subspaces, contained in the graphs of two (primal and dual) generalized derivatives. This class of mappings includes the…
We consider the linear elliptic systems or equations in divergence form with periodically oscillating coefficients. We prove the large-scale boundary Lipschitz estimate for the weak solutions in domains satisfying the so-called…
We present a very fast algorithm for general matrix factorization of a data matrix for use in the statistical analysis of high-dimensional data via latent factors. Such data are prevalent across many application areas and generate an…
Weight decay is ubiquitous in training deep neural network architectures. Its empirical success is often attributed to capacity control; nonetheless, our theoretical understanding of its effect on the loss landscape and the set of…
Trace norm regularization is a widely used approach for learning low rank matrices. A standard optimization strategy is based on formulating the problem as one of low rank matrix factorization which, however, leads to a non-convex problem.…
Low-rank matrix factorization (MF) is an important technique in data science. The key idea of MF is that there exists latent structures in the data, by uncovering which we could obtain a compressed representation of the data. By factorizing…
Subset selection for the rank $k$ approximation of an $n\times d$ matrix $A$ offers improvements in the interpretability of matrices, as well as a variety of computational savings. This problem is well-understood when the error measure is…
It is hereby established that the set of Lipschitz functions $f:\mathcal{U}\rightarrow \mathbb{R}$ ($\mathcal{U}$ nonempty open subset of $\ell_{d}^{1}$) with maximal Clarke subdifferential contains a linear subspace of uncountable…
Substructural logics naturally support a quantitative interpretation of formulas, as they are seen as consumable resources. Distances are the quantitative counterpart of equivalence relations: they measure how much two objects are similar,…
Iterative algorithms solve problems by taking steps until a solution is reached. Models in the form of Deep Thinking (DT) networks have been demonstrated to learn iterative algorithms in a way that can scale to different sized problems at…
Motivated by Tucker tensor decomposition, this paper imposes low-rank structures to the column and row spaces of coefficient matrices in a multivariate infinite-order vector autoregression (VAR), which leads to a supervised factor model…
Several key questions remain unanswered regarding overparameterized learning models. It is unclear how (stochastic) gradient descent finds solutions that generalize well, and in particular the role of small random initializations. Matrix…
We propose a new formulation of lattice theory. It is given by a matrix form and suitable for satisfying Leibniz rule on lattice. The theory may be interpreted as a multi-flavor system. By realizing the difference operator as a commutator,…
In a capacitated directed graph, it is known that the set of all min-cuts forms a distributive lattice [1], [2]. Here, we describe this lattice as a regular predicate whose forbidden elements can be advanced in constant parallel time after…
Defining the number of latent factors has been one of the most challenging problems in factor analysis. Infinite factor models offer a solution to this problem by applying increasing shrinkage on the columns of factor loading matrices, thus…
Techniques involving factorization are found in a wide range of applications and have enjoyed significant empirical success in many fields. However, common to a vast majority of these problems is the significant disadvantage that the…