Related papers: How the Move Acceptance Hyper-Heuristic Copes With…
Long horizon lengths in Moving Horizon Estimation are desirable to reach the performance limits of the full information estimator. However, the conventional MHE technique suffers from a number of deficiencies in this respect. First, the…
Human trajectory forecasting requires capturing the multimodal nature of pedestrian behavior. However, existing approaches suffer from prior misalignment. Their learned or fixed priors often fail to capture the full distribution of…
Motivated by novel results in the theory of complex adaptive systems, we analyze the dynamics of random walks in which the jumping probabilities are {\it time-dependent}. We determine the survival probability in the presence of an absorbing…
Stochastic time-varying optimization is an integral part of learning in which the shape of the function changes over time in a non-deterministic manner. This paper considers multiple models of stochastic time variation and analyzes the…
Heuristics are commonly used to tackle various search and optimization problems. Design heuristics usually require tedious manual crafting with domain knowledge. Recent works have incorporated Large Language Models (LLMs) into automatic…
We present the first comprehensive study of latent multi-head attention (MLA) for small language models, revealing interesting efficiency-quality trade-offs. Training 30M-parameter GPT models on 100,000 synthetic stories, we benchmark three…
We develop a simple routine unifying the analysis of several important recently-developed stochastic optimization methods including SAGA, Finito, and stochastic dual coordinate ascent (SDCA). First, we show an intrinsic connection between…
When tuning software configuration for better performance (e.g., latency or throughput), an important issue that many optimizers face is the presence of local optimum traps, compounded by a highly rugged configuration landscape and…
We construct a new Markov chain Monte Carlo method on finite states with optimal choices of acceptance-rejection ratio functions. We prove that the constructed continuous time Markov jumping process has a global in-time convergence rate in…
Stochastic gradient descent is the most prevalent algorithm to train neural networks. However, other approaches such as evolutionary algorithms are also applicable to this task. Evolutionary algorithms bring unique trade-offs that are worth…
Multi-turn, multi-agent LLM game evaluations often exhibit substantial run-to-run variance. In long-horizon interactions, small early deviations compound across turns and are amplified by multi-agent coupling. This biases win rate estimates…
We consider problems in which a mobile robot samples an unknown function defined over its operating space, so as to find a global optimum of this function. The path traveled by the robot matters, since it influences energy and time…
Moving horizon estimation (MHE) offers benefits relative to other estimation approaches by its ability to explicitly handle constraints, but suffers increased computation cost. To help enable MHE on platforms with limited computation power,…
We study the adaptation dynamics of an initially maladapted asexual population with genotypes represented by binary sequences of length $L$. The population evolves in a maximally rugged fitness landscape with a large number of local optima.…
Highway on-ramp merging is of great challenge for autonomous vehicles (AVs), since they have to proactively interact with surrounding vehicles to enter the main road safely within limited time. However, existing decision-making algorithms…
Adaptive optimization methods are well known to achieve superior convergence relative to vanilla gradient methods. The traditional viewpoint in optimization, particularly in convex optimization, explains this improved performance by arguing…
Hierarchical optimization refers to problems with interdependent decision variables and objectives, such as minimax and bilevel formulations. While various algorithms have been proposed, existing methods and analyses lack adaptivity in…
Accurately simulating the non-Markovian dynamics of open quantum systems remains a significant challenge. While the recently proposed time-evolving matrix product operator (TEMPO) algorithm based on path integrals successfully circumvents…
Adiabatic state preparation provides an analytical solution for generating the ground state of a target Hamiltonian, starting from an easily prepared ground state of the initial Hamiltonian. While effective for time-dependent Hamiltonians…
In optimal quantum-mechanical evolutions, motion can take place along paths of minimal length within an optimal time frame. Alternatively, optimal evolutions may occur along established paths without any waste of energy resources and…