Related papers: Softmax parameterization of the occupation numbers…
The leading terms in the large-$R$ asymptotics of the functional of the one-electron reduced density matrix for the ground-state energy of the H$_2$ molecule with the internuclear separation $R$ is derived thanks to the solution of the…
Black-box policy optimization is a class of reinforcement learning algorithms that explores and updates the policies at the parameter level. This class of algorithms is widely applied in robotics with movement primitives or…
In this paper, we consider optimizing a smooth, convex, lower semicontinuous function in Riemannian space with constraints. To solve the problem, we first convert it to a dual problem and then propose a general primal-dual algorithm to…
This paper considers the problem of minimizing the ordered weighted average (or ordered median) function of finitely many rational functions over compact semi-algebraic sets. Ordered weighted averages of rational functions are not, in…
In state-of-the-art deep learning for object recognition, SoftMax and Sigmoid functions are most commonly employed as the predictor outputs. Such layers often produce overconfident predictions rather than proper probabilistic scores, which…
The convergence rate of various first-order optimization algorithms is a pivotal concern within the numerical optimization community, as it directly reflects the efficiency of these algorithms across different optimization problems. Our…
The softmax activation function plays a crucial role in the success of large language models (LLMs), particularly in the self-attention mechanism of the widely adopted Transformer architecture. However, the underlying learning dynamics that…
The occupation of d-orbitals controls the magnitude and anisotropy of the inter-atomic electron transfer in transition metal oxides and hence exerts a key influence on their chemical bonding and physical properties. Atomic-scale modulations…
The class of transition metal compounds shows an enormous richness of physical properties, such as metal-insulator transitions, colossal magneto-resistance, super-conductivity, magneto-optics and spin-depend transport. It now becomes more…
We introduce a new approximate solution technique for first-order Markov decision processes (FOMDPs). Representing the value function linearly w.r.t. a set of first-order basis functions, we compute suitable weights by casting the…
The computational cost of training with softmax cross entropy loss grows linearly with the number of classes. For the settings where a large number of classes are involved, a common method to speed up training is to sample a subset of…
With an eye towards human-centered automation, we contribute to the development of a systematic means to infer features of human decision-making from behavioral data. Motivated by the common use of softmax selection in models of human…
Recently, a non-iterative multi-frequency subspace migration imaging algorithm was developed based on an asymptotic expansion formula for thin, curve-like electromagnetic inclusions and the structure of singular vectors in the Multi-Static…
The Softmax bottleneck was first identified in language modeling as a theoretical limit on the expressivity of Softmax-based models. Being one of the most widely-used methods to output probability, Softmax-based models have found a wide…
The research in parallel machine scheduling in combinatorial optimization suggests that the desirable parallel efficiency could be achieved when the jobs are sorted in the non-increasing order of processing times. In this paper, we find…
In this paper, we utilize results from convex analysis and monotone operator theory to derive additional properties of the softmax function that have not yet been covered in the existing literature. In particular, we show that the softmax…
Ranking tasks constitute fundamental components of extreme similarity learning frameworks, where extremely large corpora of objects are modeled through relative similarity relationships adhering to predefined ordinal structures. Among…
Symmetric submodular function minimization admits purely combinatorial algorithms using special orderings of the ground set. Extending the minimum-cut algorithm of Nagamochi and Ibaraki (1992), Queyranne (1998) showed that the maximum…
This paper introduces two new algorithms to accurately estimate the process noise covariance of a discrete-time Kalman filter online for robust orbit determination in the presence of dynamics model uncertainties. Common orbit determination…
Numerical optimal control has long been split between globally structured but dimensionally intractable Hamilton--Jacobi--Bellman (HJB) methods and scalable but local trajectory optimization. We introduce Featurized Occupation Measures…