相关论文: The Diffusion-Attention Connection

Knowledge graph reasoning plays a vital role in various applications and has garnered considerable attention. Recently, path-based methods have achieved impressive performance. However, they may face limitations stemming from constraints in…

Diffusion bridges have shown potential in paired image-to-image (I2I) translation tasks. However, existing methods are limited by their unidirectional nature, requiring separate models for forward and reverse translations. This not only…

Transformers have emerged as a powerful neural network architecture capable of tackling a wide range of learning tasks. In this work, we provide a theoretical analysis of their ability to automatically extract structure from data in an…

Schrodinger Bridges (SBs) are diffusion processes that steer, in finite time, a given initial distribution to another final one while minimizing a suitable cost functional. Although various methods for computing SBs have recently been…

We introduce a data-driven approach to building reduced dynamical models through manifold learning; the reduced latent space is discovered using Diffusion Maps (a manifold learning technique) on time series data. A second round of Diffusion…

Sampling from a distribution $p(x) \propto e^{-\mathcal{E}(x)}$ known up to a normalising constant is an important and challenging problem in statistics. Recent years have seen the rise of a new family of amortised sampling algorithms,…

Understanding the transport behavior of quantum many-body systems constitutes an important physical endeavor, both experimentally and theoretically. While a reliable classification into normal and anomalous dynamics is known to be…

We analyze the emergence of diffractive focusing in the transition from discrete to continuous space-time variables. Three types of dynamical equations are studied in a top-to-bottom approach, starting with the most general system. First we…

We find a consistency between two different approaches of hard diffraction, namely the QCD dipole model and the Soft Colour Interaction approach. A theoretical interpretation in terms of S-Matrix and perturbative QCD properties in the small…

We consider a generalization of the diffusion equation on graphs. This generalized diffusion equation gives rise to both normal and superdiffusive processes on infinite one-dimensional graphs. The generalization is based on the $k$-path…

Attention sinks -- tokens that receive disproportionate attention mass -- are assumed to be functionally important in autoregressive language models, but their role in diffusion transformers remains unclear. We present a causal analysis in…

Transformer tends to overallocate attention to irrelevant context. In this work, we introduce Diff Transformer, which amplifies attention to the relevant context while canceling noise. Specifically, the differential attention mechanism…

Transformer-based video diffusion models (VDMs) deliver state-of-the-art video generation quality but are constrained by the quadratic cost of self-attention, making long sequences and high resolutions computationally expensive. While…

In modeling phase transitions, it is useful to be able to connect diffuse interface descriptions of the dynamics with corresponding limiting sharp interface motions. In the case of the deep quench obstacle problem (DQOP) and surface…

We consider the effective surface motion of a particle that freely diffuses in the bulk and intermittently binds to that surface. From an exact approach we derive various regimes of the effective surface motion characterized by physical…

Attention architectures are widely used; they recently gained renewed popularity with Transformers yielding a streak of state of the art results. Yet, the geometrical implications of softmax-attention remain largely unexplored. In this work…

The crossover among two or more types of diffusive processes represents a vibrant theme in nonequilibrium statistical physics. In this work we propose two models to generate crossovers among different L\'evy processes: in the first model we…

Offline planning often struggles with poor sampling efficiency as it tries to learn policies from scratch. Especially with diffusion models, such cold start practices mean that both training and sampling become very expensive. We…

The, so called, "hard-scattering approach" represents a suitable framework for the perturbative treatment of exclusive hadronic processes at large energies and (transverse) momentum transfers. In this context, diquarks can serve as a useful…

Stable Diffusion fine-tuning technique is tried to assist bridge-type innovation. The bridge real photo dataset is built, and Stable Diffusion is fine tuned by using four methods that are Textual Inversion, Dreambooth, Hypernetwork and…