Related papers: Learning Complexity of Simulated Annealing

Annealing is the process of gradually lowering the temperature of a system to guide it towards its lowest energy states. In an accompanying paper [Luo et al. Phys. Rev. E 108, L052105 (2023)], we derived a general bound on annealing…

Many model selection algorithms rely on sparse dictionary learning to provide interpretable and physics-based governing equations. The optimization algorithms typically use a hard thresholding process to enforce sparse activations in the…

We present a near-optimal reduction from approximately counting the cardinality of a discrete set to approximately sampling elements of the set. An important application of our work is to approximating the partition function $Z$ of a…

Yielding behavior in amorphous solids has been investigated in computer simulations employing uniform and cyclic shear deformation. Recent results characterise yielding as a discontinuous transition, with the degree of annealing of glasses…

Given a target function $U$ to minimize on a finite state space $\mathcal{X}$, a proposal chain with generator $Q$ and a cooling schedule $T(t)$ that depends on time $t$, in this paper we study two types of simulated annealing (SA)…

We describe a solution to the student-project allocation problem using simulated annealing. The problem involves assigning students to projects, where each student has ranked a fixed number of projects in order of preference. Each project…

Population annealing is a variant of the simulated annealing algorithm that improves the quality of the thermalization process in systems with rough free-energy landscapes by introducing a resampling process. We consider the diluted…

This study combines simulated annealing with delta evaluation to solve the joint stratification and sample allocation problem. In this problem, atomic strata are partitioned into mutually exclusive and collectively exhaustive strata. Each…

Learning-to-rank is an applied domain of supervised machine learning. As feature selection has been found to be effective for improving the accuracy of learning models in general, it is intriguing to investigate this process for…

We propose a new global optimization method ({\em Simulated Tempering}) for simulating effectively a system with a rough free energy landscape (i.e. many coexisting states) at finite non-zero temperature. This method is related to simulated…

We recapitulate the Bayesian formulation of neural network based classifiers and show that, while sampling from the posterior does indeed lead to better generalisation than is obtained by standard optimisation of the cost function, even…

Annealed importance sampling (AIS) is a common algorithm to estimate partition functions of useful stochastic models. One important problem for obtaining accurate AIS estimates is the selection of an annealing schedule. Conventionally, an…

In this work we present a general machine learning based scheme to optimize experimental control. The method utilizes the neural network to learn the relation between the control parameters and the control goal, with which the optimal…

We study adaptive learning rate scheduling for norm-constrained optimizers (e.g., Muon and Lion). We introduce a generalized smoothness assumption under which local curvature decreases with the suboptimality gap and empirically verify that…

Global optimization heuristics are popular to optimize hard non-convex problems. Despite their irrefutably large cost-to-solution, in the lack of other working greedy or convex approaches, global optimization algorithms remain the…

Finding a ground state of a given Hamiltonian of an Ising model on a graph $G=(V,E)$ is an important but hard problem. The standard approach for this kind of problem is the application of algorithms that rely on single-spin-flip Markov…

Multi-sample aggregation strategies, such as majority voting and best-of-N sampling, are widely used in contemporary large language models (LLMs) to enhance predictive accuracy across various tasks. A key challenge in this process is…

Inherent in virtually every iterative machine learning algorithm is the problem of hyper-parameter tuning, which includes three major design parameters: (a) the complexity of the model, e.g., the number of neurons in a neural network, (b)…

We investigate the theoretical foundations of the simulated tempering method and use our findings to design efficient algorithms. Employing a large deviation argument first used for replica exchange molecular dynamics [Plattner et al., J.…

A Boltzmann machine whose effective "temperature" can be dynamically "cooled" provides a stochastic neural network realization of simulated annealing, which is an important metaheuristic for solving combinatorial or global optimization…