Related papers: Optimisation challenge for superconducting adiabat…

Neural network training can be accelerated when a learnable update rule is used in lieu of classic adaptive optimizers (e.g. Adam). However, learnable update rules can be costly and unstable to train and use. Recently, Jang et al. (2023)…

Expensive-to-train deep learning models can benefit from an optimization of the hyperparameters that determine the model architecture. We optimize 23 hyperparameters of a materials informatics model, Compositionally-Restricted…

In the last decade, the interest to emulation of the functionality and structure of the human brain to solve the problems related to image processing and pattern recognition, especially using to Artificial Neural Network (ANN), has…

This paper addresses a distributed optimization problem in a communication network where nodes are active sporadically. Each active node applies some learning method to control its action to maximize the global utility function, which is…

Deep Neural Networks are powerful tools for solving machine learning problems, but their training often involves dense and costly parameter updates. In this work, we use a novel Max-Plus neural architecture in which classical addition and…

We consider the fundamental problem of learning a single neuron $x \mapsto\sigma(w^\top x)$ using standard gradient methods. As opposed to previous works, which considered specific (and not always realistic) input distributions and…

This paper shows that a new type of artificial neural network (ANN) -- the Simultaneous Recurrent Network (SRN) -- can, if properly trained, solve a difficult function approximation problem which conventional ANNs -- either feedforward or…

Neuromorphic computing, inspired by the brain, promises extreme efficiency for certain classes of learning tasks, such as classification and pattern recognition. The performance and power consumption of neuromorphic computing depends…

We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments. The method is straightforward to implement, is computationally efficient, has…

Adaptive gradient-based optimizers such as Adagrad and Adam are crucial for achieving state-of-the-art performance in machine translation and language modeling. However, these methods maintain second-order statistics for each parameter,…

We implement a classical counterpart of a scalable design for adiabatic quantum computation. The key element of this design is a coupler providing controllable coupling between two bistable elements (in our case superconducting rings with a…

Artificial neural networks have gone through a recent rise in popularity, achieving state-of-the-art results in various fields, including image classification, speech recognition, and automated control. Both the performance and…

Existing high-performance deep learning models require very intensive computing. For this reason, it is difficult to embed a deep learning model into a system with limited resources. In this paper, we propose the novel idea of the network…

Modeling high-dimensional time series with simple structures is a challenging problem. This paper proposes a network double autoregression (NDAR) model, which combines the advantages of network structure and the double autoregression (DAR)…

In this paper, we discuss a different type of semi-supervised setting: a coarse level of labeling is available for all observations but the model has to learn a fine level of latent annotation for each one of them. Problems in this setting…

Neuromorphic networks of artificial neurons and synapses can solve computational hard problems with energy efficiencies unattainable for von Neumann architectures. For image processing, silicon neuromorphic processors outperform graphic…

In this work, we propose an optimization algorithm which we call norm-adapted gradient descent. This algorithm is similar to other gradient-based optimization algorithms like Adam or Adagrad in that it adapts the learning rate of stochastic…

Performing machine learning with analog signals offers advantages in speed and energy efficiency, but sensitivity to component and measurement imperfections often foils training without a system-specific companion digital model. Here we…

Optimal actuator and control design is studied as a multi-level optimisation problem, where the actuator design is evaluated based on the performance of the associated optimal closed loop. The evaluation of the optimal closed loop for a…

Dendrites are crucial structures for computation of an individual neuron. It has been shown that the dynamics of a biological neuron with dendrites can be approximated by artificial neural networks (ANN) with deep structure. However, it…