Related papers: When Less is More: Approximating the Quantum Geome…
We propose an efficient method for approximating natural gradient descent in neural networks which we call Kronecker-Factored Approximate Curvature (K-FAC). K-FAC is based on an efficiently invertible approximation of a neural network's…
Variational quantum algorithms are promising tools whose efficacy depends on their optimisation method. For noise-free unitary circuits, the quantum generalisation of natural gradient descent has been introduced and shown to be equivalent…
This work proposes a time-efficient Natural Gradient Descent method, called TENGraD, with linear convergence guarantees. Computing the inverse of the neural network's Fisher information matrix is expensive in NGD because the Fisher matrix…
A quantum generalization of Natural Gradient Descent is presented as part of a general-purpose optimization framework for variational quantum circuits. The optimization dynamics is interpreted as moving in the steepest descent direction…
Variational quantum algorithms are promising tools for near-term quantum computers as their shallow circuits are robust to experimental imperfections. Their practical applicability, however, strongly depends on how many times their circuits…
Natural gradient is an advanced optimization method based on information geometry, where the Fisher metric plays a crucial role. Its quantum counterpart, known as quantum natural gradient (QNG), employs the symmetric logarithmic derivative…
The efficient simulation of complex quantum systems remains a central challenge due to the exponential growth of Hilbert space with system size. Tensor network methods have long been established as powerful approximation schemes, and their…
The natural gradient descent optimisation technique is an efficient optimising protocol for broad classes of classical and quantum systems that takes the underlying geometry of the parameter manifold into account by means of using either…
Quantum natural gradient has emerged as a superior minimisation technique in quantum variational algorithms. Classically simulating the algorithm running on near-future quantum hardware is paramount in its study, as it is for all…
A deep neural network is a hierarchical nonlinear model transforming input signals to output signals. Its input-output relation is considered to be stochastic, being described for a given input by a parameterized conditional probability…
Training neural networks with many processors can reduce time-to-solution; however, it is challenging to maintain convergence and efficiency at large scales. The Kronecker-factored Approximate Curvature (K-FAC) was recently proposed as an…
In the context of deep learning, many optimization methods use gradient covariance information in order to accelerate the convergence of Stochastic Gradient Descent. In particular, starting with Adagrad, a seemingly endless line of research…
The representation of a quantum wave function as a neural network quantum state (NQS) provides a powerful variational ansatz for finding the ground states of many-body quantum systems. Nevertheless, due to the complex variational landscape,…
Variational quantum algorithms, optimized using gradient-based methods, often exhibit sub-optimal convergence performance due to their dependence on Euclidean geometry. Quantum natural gradient descent (QNGD) is a more efficient method that…
Natural Gradient Descent, a second-degree optimization method motivated by the information geometry, makes use of the Fisher Information Matrix instead of the Hessian which is typically used. However, in many cases, the Fisher Information…
Second-order methods for neural network optimization have several advantages over methods based on first-order gradient descent, including better scaling to large mini-batch sizes and fewer updates needed for convergence. But they are…
Variational Quantum algorithms, especially Quantum Approximate Optimization and Variational Quantum Eigensolver (VQE) have established their potential to provide computational advantage in the realm of combinatorial optimization. However,…
The tomographic picture of quantum mechanics has brought the description of quantum states closer to that of classical probability and statistics. On the other hand, the geometrical formulation of quantum mechanics introduces a metric…
As a second-order method, the Natural Gradient Descent (NGD) has the ability to accelerate training of neural networks. However, due to the prohibitive computational and memory costs of computing and inverting the Fisher Information Matrix…
Due to the multi-linearity of tensors, most algorithms for tensor optimization problems are designed based on the block coordinate descent method. Such algorithms are widely employed by practitioners for their implementability and…