Related papers: Adaptive, Fast and Oblivious Convolution in Evolut…

This paper presents a new technique for deterministic length reduction. This technique improves the running time of the algorithm presented in \cite{LR07} for performing fast convolution in sparse data. While the regular fast convolution of…

We present a novel algorithm attaining excessively fast, the sought solution of linear systems of equations. The algorithm is short in its basic formulation and, by definition, vectorized, while the memory allocation demands are trivial,…

Drawing inspiration from gradient-based meta-learning methods with infinitely small gradient steps, we introduce Continuous-Time Meta-Learning (COMLN), a meta-learning algorithm where adaptation follows the dynamics of a gradient vector…

Evolutionary computation methods have been successfully applied to neural networks since two decades ago, while those methods cannot scale well to the modern deep neural networks due to the complicated architectures and large quantities of…

Real-world optimization problems often involve stochastic and dynamic components. Evolutionary algorithms are particularly effective in these scenarios, as they can easily adapt to uncertain and changing environments but often uncertainty…

Models invoking the chemical master equation are used in many areas of science, and, hence, their simulation is of interest to many researchers. The complexity of the problems at hand often requires considerable computational power, so a…

Deep neural networks have excelled on a wide range of problems, from vision to language and game playing. Neural networks very gradually incorporate information into weights as they process data, requiring very low learning rates. If the…

We assume the permutation $\pi$ is given by an $n$-element array in which the $i$-th element denotes the value $\pi(i)$. Constructing its inverse in-place (i.e. using $O(\log{n})$ bits of additional memory) can be achieved in linear time…

Feature selection is one of the most challenging issues in machine learning, especially while working with high dimensional data. In this paper, we address the problem of feature selection and propose a new approach called Evolving Fast and…

In stochastic convex optimization problems, most existing adaptive methods rely on prior knowledge about the diameter bound $D$ when the smoothness or the Lipschitz constant is unknown. This often significantly affects performance as only a…

The evolution of cognition is frequently discussed as the evolution of cognitive abilities or the evolution of some neuronal structures in the brain. However, since such traits or abilities are often highly complex, understanding their…

In the paper, we present a high order fast algorithm with almost optimum memory for the Caputo fractional derivative, which can be expressed as a convolution of $u'(t)$ with the kernel $(t_n-t)^{-\alpha}$. In the fast algorithm, the…

Animals behave adaptively in the environment with multiply competing goals. Understanding of the mechanisms underlying such goal-directed behavior remains a challenge for neuroscience as well for adaptive system research. To address this…

Current phylogenetic comparative methods generally employ the Ornstein-Uhlenbeck(OU) process for modeling trait evolution. Being able of tracking the optimum of a trait within a group of related species, the OU process provides information…

We investigate the well-posedness and solution regularity of an evolution equation with non-positive type variable-exponent memory, which describes multiscale viscoelasticity in materials with memory. The perturbation method is applied for…

This short survey article stems from recent progress on critical cases of stochastic evolution equations in variational formulation with additive, multiplicative or gradient noises. Typical examples appear as the limit cases of the…

Adaptive gradient-based optimizers such as Adagrad and Adam are crucial for achieving state-of-the-art performance in machine translation and language modeling. However, these methods maintain second-order statistics for each parameter,…

Large Transformer models routinely achieve state-of-the-art results on a number of tasks but training these models can be prohibitively costly, especially on long sequences. We introduce two techniques to improve the efficiency of…

Adaptive sampling algorithms are modern and efficient methods that dynamically adjust the sample size throughout the optimization process. However, they may encounter difficulties in risk-averse settings, particularly due to the challenge…

Selecting an effective step-size is a fundamental challenge in first-order optimization, especially for problems with non-Euclidean geometries. This paper presents a novel adaptive step-size strategy for optimization algorithms that rely on…