Related papers: Dynamic Mode Decomposition along Depth in Vision T…

Dynamic Mode Decomposition (DMD) is a data-driven modal decomposition technique that extracts coherent spatio-temporal structures from high-dimensional time-series data. By decomposing the dynamics into a set of modes, each associated with…

This paper builds on the theoretical foundations for dynamic mode decomposition (DMD) of control-affine dynamical systems by leveraging the theory of vector-valued reproducing kernel Hilbert spaces (RKHSs). Specifically, control Liouville…

Vision transformers (ViTs) - especially feature foundation models like DINOv2 - learn rich representations useful for many downstream tasks. However, architectural choices (such as positional encoding) can lead to these models displaying…

Vision transformers (ViTs) have demonstrated remarkable performance in a variety of vision tasks. Despite their promising capabilities, training a ViT requires a large amount of diverse data. Several studies empirically found that using…

Aero-optical beam control relies on the development of low-latency forecasting techniques to quickly predict wavefronts aberrated by the Turbulent Boundary Layer (TBL) around an airborne optical system, and its study applies to a…

Attention is sparse in vision transformers. We observe the final prediction in vision transformers is only based on a subset of most informative tokens, which is sufficient for accurate image recognition. Based on this observation, we…

For computer vision, Vision Transformers (ViTs) have become one of the go-to deep net architectures. Despite being inspired by Convolutional Neural Networks (CNNs), ViTs' output remains sensitive to small spatial shifts in the input, i.e.,…

Dynamic Mode Decomposition (DMD) has become synonymous with the Koopman operator, where continuous time dynamics are examined through a discrete time proxy determined by a fixed timestep using Koopman (i.e. composition) operators. Using the…

Dynamic Mode Decomposition (DMD) is a technique to approximate generally non-linear dynamical systems using linear techniques, which are better understood and easier to analyze. Koopman theory extends DMD by transforming the original system…

Vision Transformers (ViTs) can learn strong image-level representations while their patch representations become less effective for dense prediction during prolonged training. We revisit this dense degradation phenomenon and argue that it…

We present a low-rank Koopman operator formulation for accelerating deformable subspace simulation. Using a Dynamic Mode Decomposition (DMD) parameterization of the Koopman operator, our method learns the temporal evolution of deformable…

Existing parameter-efficient fine-tuning (PEFT) methods have achieved significant success on vision transformers (ViTs) adaptation by improving parameter efficiency. However, the exploration of enhancing inference efficiency during…

Vision Transformer (ViT) and its variants (e.g., Swin, PVT) have achieved great success in various computer vision tasks, owing to their capability to learn long-range contextual information. Layer Normalization (LN) is an essential…

This work aims to improve the efficiency of vision transformers (ViT). While ViTs use computationally expensive self-attention operations in every layer, we identify that these operations are highly correlated across layers -- a key…

The content-agnostic, fixed-grid tokenizers used by standard large-scale vision models like Vision Transformer (ViT) and Vision Mamba (Vim) represent a fundamental performance bottleneck, creating a trade-off between capturing fine-grained…

Vision Transformers (ViTs) have achieved state-of-the-art performance on various vision tasks. However, ViTs' self-attention module is still arguably a major bottleneck, limiting their achievable hardware efficiency. Meanwhile, existing…

The Dynamic Mode Decomposition (DMD) is a Koopman-based algorithm that straightforwardly isolates individual mechanisms from the compound morphology of direct measurement. However, many may be perplexed by the messages the DMD structures…

Dynamic MRI reconstruction from undersampled measurements is a challenging inverse problem that requires preserving both spatial reconstruction quality and temporal consistency across the frames of the cine series. While recent…

Vision transformer (ViT) models exhibit substandard optimizability. In particular, they are sensitive to the choice of optimizer (AdamW vs. SGD), optimizer hyperparameters, and training schedule length. In comparison, modern convolutional…

Vision transformers (ViTs) have gained increasing popularity as they are commonly believed to own higher modeling capacity and representation flexibility, than traditional convolutional networks. However, it is questionable whether such…