Related papers: Accelerating Feedforward Computation via Parallel …
Recently Ahmadi et al. (2021) and Tagliaferro (2022) proposed some iterative methods for the numerical solution of linear systems which, under the classical hypothesis of strict diagonal dominance, typically converge faster than the Jacobi…
Large autoregressive models can generate high-quality, high-resolution images but suffer from slow generation speed, because these models require hundreds to thousands of sequential forward passes for next-token prediction during inference.…
The majority of methods used to compute approximations to the Hamilton-Jacobi-Isaacs partial differential equation (HJI PDE) rely on the discretization of the state space to perform dynamic programming updates. This type of approach is…
Transformers and linear state space models can be evaluated in parallel on modern hardware, but evaluating nonlinear RNNs appears to be an inherently sequential problem. Recently, however, Lim et al. '24 developed an approach called DEER,…
As has been shown in our previous work, the parallel-in-time direct inverse (ParaDIn) method introduced by Yamaleev and Paudel in (arXiv: 2406.00878v1, 2024) imposes some constraint on the maximum number of time levels, $N_t$, that can be…
The main goal of distribution network (DN) expansion planning is essentially to achieve minimal investment constrained with specified reliability requirements. The reliability-constrained distribution network planning (RcDNP) problem can be…
The current large auto-regressive models can generate high-quality, high-resolution images, but these models require hundreds or even thousands of steps of next-token prediction during inference, resulting in substantial time consumption.…
We consider the problem of computing reachability probabilities: given a Markov chain, an initial state of the Markov chain, and a set of goal states of the Markov chain, what is the probability of reaching any of the goal states from the…
The regression of a functional response on a set of scalar predictors can be a challenging task, especially if there is a large number of predictors, or the relationship between those predictors and the response is nonlinear. In this work,…
The Frank Wolfe algorithm (FW) is a popular projection-free alternative for solving large-scale constrained optimization problems. However, the FW algorithm suffers from a sublinear convergence rate when minimizing a smooth convex function…
In parallel simulation, convergence and parallelism are often seen as inherently conflicting objectives. Improved parallelism typically entails lighter local computation and weaker coupling, which unavoidably slow the global convergence.…
A control theoretic approach is presented in this paper for both batch and instantaneous updates of weights in feed-forward neural networks. The popular Hamilton-Jacobi-Bellman (HJB) equation has been used to generate an optimal weight…
Quantum machine learning has the potential for broad industrial applications, and the development of quantum algorithms for improving the performance of neural networks is of particular interest given the central role they play in machine…
The paper is devoted to the development of a methodology for evaluating the scalability of compute-intensive iterative algorithms used in simulating complex physical processes on supercomputer systems. The proposed methodology is based on…
The Jacobi prior offers an alternative Bayesian framework, designed to achieve superior computational efficiency without compromising predictive performance. Compared to widely used methods such as Lasso, Ridge, Elastic Net, uniLasso, the…
Backpropagation algorithm is indispensable for the training of feedforward neural networks. It requires propagating error gradients sequentially from the output layer all the way back to the input layer. The backward locking in…
In this paper we introduce an iterative Jacobi algorithm for solving distributed model predictive control (DMPC) problems, with linear coupled dynamics and convex coupled constraints. The algorithm guarantees stability and persistent…
This paper introduces a new architectural framework, known as input fast-forwarding, that can enhance the performance of deep networks. The main idea is to incorporate a parallel path that sends representations of input values forward to…
Recurrent neural network architectures can have useful computational properties, with complex temporal dynamics and input-sensitive attractor states. However, evaluation of recurrent dynamic architectures requires solution of systems of…
A promising trend in deep learning replaces traditional feedforward networks with implicit networks. Unlike traditional networks, implicit networks solve a fixed point equation to compute inferences. Solving for the fixed point varies in…