Related papers: Modeling Nonlinear Dynamics from Videos

In this paper, we introduce a modular deep neural network (DNN) framework for data-driven reduced order modeling of dynamical systems relevant to fluid flows. We propose various deep neural network architectures which numerically predict…

Traditional SLAM systems, which rely on bundle adjustment, struggle with highly dynamic scenes commonly found in casual videos. Such videos entangle the motion of dynamic elements, undermining the assumption of static environments required…

We propose a new approach to learn to segment multiple image objects without manual supervision. The method can extract objects form still images, but uses videos for supervision. While prior works have considered motion for segmentation, a…

This article discusses a newly developed online manifold learning method, subspace iteration using reduced models (SIRM), for the dimensionality reduction of dynamical systems. This method may be viewed as subspace iteration combined with a…

Observable operator models (OOMs) and related models are one of the most important and powerful tools for modeling and analyzing stochastic systems. They exactly describe dynamics of finite-rank systems and can be efficiently and…

We propose an automatic system for organizing the content of a collection of unstructured videos of an articulated object class (e.g. tiger, horse). By exploiting the recurring motion patterns of the class across videos, our system: 1)…

We study the fundamental problem of learning a marginally stable unknown nonlinear dynamical system. We describe an algorithm for this problem, based on the technique of spectral filtering, which learns a mapping from past observations to…

Unsupervised multi-object segmentation has shown impressive results on images by utilizing powerful semantics learned from self-supervised pretraining. An additional modality such as depth or motion is often used to facilitate the…

Balancing temporal resolution and spatial detail under limited compute budget remains a key challenge for video-based multi-modal large language models (MLLMs). Existing methods typically compress video representations using predefined…

This paper introduces a SSSUMO, semi-supervised deep learning approach for submovement decomposition that achieves state-of-the-art accuracy and speed. While submovement analysis offers valuable insights into motor control, existing methods…

We present a method for unsupervised learning of equations of motion for objects in raw and optionally distorted unlabeled video. We first train an autoencoder that maps each video frame into a low-dimensional latent space where the laws of…

Slow motion videos are becoming increasingly popular, but capturing high-resolution videos at extremely high frame rates requires professional high-speed cameras. To mitigate this problem, current techniques increase the frame rate of…

We present a data-driven nonintrusive model order reduction method for dynamical systems with moving boundaries. The proposed method draws on the proper orthogonal decomposition, Gaussian process regression, and moving least squares…

Although commercial and open-source software exist to reconstruct a static object from a sequence recorded with an RGB-D sensor, there is a lack of tools that build rigged models of articulated objects that deform realistically and can be…

We propose a new technique for obtaining reduced order models for nonlinear dynamical systems. Specifically, we advocate the use of the recently developed Dynamic Mode Decomposition (DMD), an equation-free method, to approximate the…

Isolating different types of motion in video data is a highly relevant problem in video analysis. Applications can be found, for example, in dynamic medical or biological imaging, where the analysis and further processing of the dynamics of…

Piecewise-linear nonlinear systems appear in many engineering disciplines. Prediction of the dynamic behavior of such systems is of great importance from practical and theoretical viewpoint. In this paper, a data-driven model order…

Data-driven dimensionality reduction methods such as proper orthogonal decomposition (POD) and dynamic mode decomposition (DMD) have proven to be useful for exploring complex phenomena within fluid dynamics and beyond. A well-known…

Model order reduction techniques simplify high-dimensional dynamical systems by deriving lower-dimensional models that retain essential system characteristics. These techniques are crucial for the controller design of complex systems while…

Multiple time scale stochastic dynamical systems are ubiquitous in science and engineering, and the reduction of such systems and their models to only their slow components is often essential for scientific computation and further analysis.…