Related papers: The Mori-Zwanzig formulation of deep learning
We present a general numerical approach for constructing governing equations for unknown dynamical systems when only data on a subset of the state variables are available. The unknown equations for these observed variables are thus a…
We develop rigorous estimates and provably convergent approximations for the memory integral in the Mori-Zwanzig (MZ) formulation. The new theory is built upon rigorous mathematical foundations and is presented for both state-space and…
The well-known Mori-Zwanzig theory tells us that model reduction leads to memory effect. For a long time, modeling the memory effect accurately and efficiently has been an important but nearly impossible task in developing a good reduced…
In many time-dependent problems of practical interest the parameters and/or initial conditions entering the equations describing the evolution of the various quantities exhibit uncertainty. One way to address the problem of how this…
We propose a novel learning framework based on neural mean-field dynamics for inference and estimation problems of diffusion on networks. Our new framework is derived from the Mori-Zwanzig formalism to obtain an exact evolution of the node…
The Mori-Zwanzig projection operator formalism is one of the central tools of nonequilibrium statistical mechanics, allowing to derive macroscopic equations of motion from the microscopic dynamics through a systematic coarse-graining…
We introduce the Mori-Zwanzig Mode Decomposition (MZMD), a novel data-driven technique for efficient modal analysis of and reduced-order modeling of large-scale spatio-temporal dynamical systems. MZMD represents an extension of Dynamic Mode…
We describe a paradigm for multiscale modeling that combines the Mori-Zwanzig (MZ) formalism of Statistical Mechanics with the Variational Multiscale (VMS) method. The MZ-VMS approach leverages both VMS scale-separation projectors as well…
Explaining the emergence of stochastic irreversible macroscopic dynamics from time-reversible deterministic microscopic dynamics is one of the key problems in philosophy of physics. The Mori-Zwanzig projection operator formalism, which is…
We examine the challenging problem of constructing reduced models for the long time prediction of systems where there is no timescale separation between the resolved and unresolved variables. In previous work we focused on the case where…
In a recent preprint (arXiv:1211.4285v1) we addressed the problem of constructing reduced models for time-dependent systems described by differential equations which involve uncertain parameters. In the current work, we focus on the…
Deep learning is formulated as a discrete-time optimal control problem. This allows one to characterize necessary conditions for optimality and develop training algorithms that do not rely on gradients with respect to the trainable…
Deep learning has arguably achieved tremendous success in recent years. In simple words, deep learning uses the composition of many nonlinear functions to model the complex dependency between input features and labels. While neural networks…
Mathematical approaches from dynamical systems theory are used in a range of fields. This includes biology where they are used to describe processes such as protein-protein interaction and gene regulatory networks. As such networks increase…
Based on its great successes in inference and denosing tasks, Dictionary Learning (DL) and its related sparse optimization formulations have garnered a lot of research interest. While most solutions have focused on single layer…
We propose a novel learning framework using neural mean-field (NMF) dynamics for inference and estimation problems on heterogeneous diffusion networks. Our new framework leverages the Mori-Zwanzig formalism to obtain an exact evolution…
Built upon the hypoelliptic analysis of the effective Mori-Zwanzig (EMZ) equation for observables of stochastic dynamical systems, we show that the obtained semigroup estimates for the EMZ equation can be used to drive prior estimates of…
By bridging deep networks and physics, the programme of geometrization of deep networks was proposed as a framework for the interpretability of deep learning systems. Following this programme we can apply two key ideas of physics, the…
We discuss some mathematical aspects of the Mori-Zwanzig projection operator formalism. The core of the Mori-Zwanzig formalism is the generalised Langevin equation, which is typically derived from the Dyson-Duhamel identity. We derive the…
Deep learning achieves remarkable generalization capability with overwhelming number of model parameters. Theoretical understanding of deep learning generalization receives recent attention yet remains not fully explored. This paper…