Related papers: Training and synchronizing oscillator networks wit…
The classification of anomalies or sudden changes in power networks versus normal abrupt changes or switching actions is essential to take appropriate maintenance actions that guarantee the quality of power delivery. This issue has…
Optical approaches have made great strides towards the goal of high-speed, energy-efficient computing necessary for modern deep learning and AI applications. Read-in and read-out of data, however, limit the overall performance of existing…
Dynamical Ising machines are actively investigated from the perspective of finding efficient heuristics for NP-hard optimization problems. However, the existing data demonstrate super-polynomial scaling of the running time with the system…
Large machine learning models are revolutionary technologies of artificial intelligence whose bottlenecks include huge computational expenses, power, and time used both in the pre-training and fine-tuning process. In this work, we show that…
Gradient-based iterative optimization methods are the workhorse of modern machine learning. They crucially rely on careful tuning of parameters like learning rate and momentum. However, one typically sets them using heuristic approaches…
Large-scale distributed training of Deep Neural Networks (DNNs) on state-of-the-art platforms is expected to be severely communication constrained. To overcome this limitation, numerous gradient compression techniques have been proposed and…
Interval Bound Propagation (IBP) is so far the base of state-of-the-art methods for training neural networks with certifiable robustness guarantees when potential adversarial perturbations present, while the convergence of IBP training…
A recent line of research has shown that gradient-based algorithms with random initialization can converge to the global minima of the training loss for over-parameterized (i.e., sufficiently wide) deep neural networks. However, the…
A coherent Ising machine (CIM) is a network of optical parametric oscillators (OPOs), in which the "strongest" collective mode of oscillation at well above threshold corresponds to an optimum solution of a given Ising problem. When a pump…
Quantized Neural Networks (QNNs) are often used to improve network efficiency during the inference phase, i.e. after the network has been trained. Extensive research in the field suggests many different quantization schemes. Still, the…
Neurons in the brain are wired into adaptive networks that exhibit a range of collective dynamics. Oscillations, for example, are paradigmatic synchronous patterns of neural activity with a defined temporal scale. Neuronal avalanches, in…
Nonlinear conjugate gradient (NLCG) based optimizers have shown superior loss convergence properties compared to gradient descent based optimizers for traditional optimization problems. However, in Deep Neural Network (DNN) training, the…
Oscillatory activity is ubiquitous in natural and engineered network systems. The interaction scheme underlying interdependent oscillatory components governs the emergence of network-wide patterns of synchrony that regulate and enable…
Part I of this work [Gao25] establishes online scaled gradient methods (OSGM), a framework that utilizes online convex optimization to adapt stepsizes in gradient methods. This paper focuses on the practical aspects of OSGM. We leverage the…
SAGRAD (Simulated Annealing GRADient), a Fortran 77 program for computing neural networks for classification using batch learning, is discussed. Neural network training in SAGRAD is based on a combination of simulated annealing and…
From the flashes of fireflies to Josephson junctions and power infrastructure, networks of coupled phase oscillators provide a powerful framework to describe synchronization phenomena in many natural and engineered systems. Most real-world…
Distributed learning is widely used for training large models on large datasets by distributing parts of the model or dataset across multiple devices and aggregating the computed results for subsequent computations or parameter updates.…
Object selection refers to the mechanism of extracting objects of interest while ignoring other objects and background in a given visual scene. It is a fundamental issue for many computer vision and image analysis techniques and it is still…
First-principles-based modelings have been extremely successful in providing crucial insights and predictions for complex biological functions and phenomena. However, they can be hard to build and expensive to simulate for complex living…
Much of studies on neural computation are based on network models of static neurons that produce analog output, despite the fact that information processing in the brain is predominantly carried out by dynamic neurons that produce discrete…