Instability of explicit time integration for strongly quenched dynamics with neural quantum states

Neural quantum states have recently demonstrated significant potential for simulating quantum dynamics beyond the capabilities of existing variational ans\"{a}tze. However, studying strongly driven quantum dynamics with neural networks has proven challenging so far. Here, we focus on assessing several sources of numerical instabilities that can appear in the simulation of quantum dynamics based on the time-dependent variational principle (TDVP) with the computationally efficient explicit time integration scheme. Focusing on the restricted Boltzmann machine architecture, we compare solutions obtained by TDVP with analytical solutions and implicit methods as a function of the quench strength. Interestingly, we uncover a quenching strength that leads to a numerical breakdown in the absence of Monte Carlo noise, despite the fact that physical observables don't exhibit irregularities. This breakdown phenomenon appears consistently across several different TDVP formulations, even those that eliminate small eigenvalues of the Fisher matrix or use geometric properties to recast the equation of motion. We provide evidence that the nature of the instability stems from stiffness of the dynamics of the variational parameters, despite the absence of stiffness in the exact quantum dynamics. We conclude that alternative methods need to be developed to leverage the computational efficiency of explicit time integration of the TDVP equations for simulating strongly nonequilibrium quantum dynamics with neural-network quantum states.