Related papers: Seasons's Greetings by AD
Adaptive gradient methods, especially Adam-type methods (such as Adam, AMSGrad, and AdaBound), have been proposed to speed up the training process with an element-wise scaling term on learning rates. However, they often generalize poorly…
We present a powerful and easy-to-implement iterative algorithm for solving large-scale optimization problems that involve $L_1$/total-variation (TV) regularization. The method is based on combining the Alternating Directions Method of…
This paper establishes an analogue of the Robinson--Schensted correspondence for cylindric tableaux. In particular, for any pair of positive integers $(d,L)$, we construct a bijection between permutations that avoid the patterns $d\cdots 1…
Gradient temporal difference (Gradient TD) algorithms are a popular class of stochastic approximation (SA) algorithms used for policy evaluation in reinforcement learning. Here, we consider Gradient TD algorithms with an additional heavy…
We describe an implicit general relativistic hydrodynamics code. The evolution equations are formulated in comoving coordinates. A conservative finite differencing of the Einstein equations is outlined, and artificial viscosity and…
By coordinating terminal smart devices or microprocessors to engage in cooperative computation to achieve systemlevel targets, distributed optimization is incrementally favored by both engineering and computer science. The well-known…
The application of operator overloading algorithmic differentiation (AD) to computer programs in order to compute the derivative is quite common. But, the replacement of the underlying computational floating point type with the specialized…
For a given graph G and integers b,f >= 0, let S be a subset of vertices of G of size b+1 such that the subgraph of G induced by S is connected and S can be separated from other vertices of G by removing f vertices. We prove that every…
Agent-based models (ABMs) simulate complex systems by capturing the bottom-up interactions of individual agents comprising the system. Many complex systems of interest, such as epidemics or financial markets, involve thousands or even…
Coded distributed computation has become common practice for performing gradient descent on large datasets to mitigate stragglers and other faults. This paper proposes a novel algorithm that encodes the partial derivatives themselves and…
In this paper, we present an involution on some kind of colored $k$-ary trees which provides a combinatorial proof of a combinatorial sum involving the generalized Catalan numbers $C_{k,\gamma}(n)=\frac{\gamma}{k n+\gamma}{k n+\gamma\choose…
SGD (Stochastic Gradient Descent) is a popular algorithm for large scale optimization problems due to its low iterative cost. However, SGD can not achieve linear convergence rate as FGD (Full Gradient Descent) because of the inherent…
Machine learning and neural network models in particular have been improving the state of the art performance on many artificial intelligence related tasks. Neural network models are typically implemented using frameworks that perform…
For a positive integer sequence $\boldsymbol{a}=(a_1, \dots, a_{N+1})$, Sylvester's denumerant $E(\boldsymbol{a}; t)$ counts the number of nonnegative integer solutions to $\sum_{i=1}^{N+1} a_i x_i = t$ for a nonnegative integer $t$. It has…
We introduce Newton-ADMM, a method for fast conic optimization. The basic idea is to view the residuals of consecutive iterates generated by the alternating direction method of multipliers (ADMM) as a set of fixed point equations, and then…
Uniform sampling from the set $\mathcal{G}(\mathbf{d})$ of graphs with a given degree-sequence $\mathbf{d} = (d_1, \dots, d_n) \in \mathbb N^n$ is a classical problem in the study of random graphs. We consider an analogue for temporal…
Traditional Time-series Anomaly Detection (TAD) methods often struggle with the composite nature of complex time-series data and a diverse array of anomalies. We introduce TADNet, an end-to-end TAD model that leverages Seasonal-Trend…
We propose a new method for obtaining complete asymptotic expansions in a systematic manner, which is suitable for counting sequences of various graph families in dense regime. The core idea is to encode the two-dimensional array of…
Distributed gradient descent (DGD) is an efficient way of implementing gradient descent (GD), especially for large data sets, by dividing the computation tasks into smaller subtasks and assigning to different computing servers (CSs) to be…
We develop the third-order adaptive Adams-Bashforth time stepping and the second-order finite difference equation for variable time steps. We incorporate these schemes in the Celeris Advent software to discretize and solve the 2D extended…