Related papers: Optimizing a Transformer-based network for a deep …
We present a novel probabilistic deep learning approach, the 'Stochastic Latent Transformer' (SLT), designed for the efficient reduced-order modelling of stochastic partial differential equations. Stochastically driven flow models are…
The disturbance storm time (Dst) index is an important and useful measurement in space weather research. It has been used to characterize the size and intensity of a geomagnetic storm. A negative Dst value means that the Earth's magnetic…
Spiking neural networks (SNNs) have made great progress on both performance and efficiency over the last few years,but their unique working pattern makes it hard to train a high-performance low-latency SNN.Thus the development of SNNs still…
Noise is one of the primary sources of interference in seismic exploration. Many authors have proposed various methods to remove noise from seismic data; however, in the face of strong noise conditions, satisfactory results are often not…
In current seismic acquisition practice, there is an increasing drive for sparsely (in space) acquired data, often in irregular geometry. These surveys can trade off subsurface information for efficiency/cost - creating a problem of…
Numerical simulations of seismic wave propagation are crucial for investigating velocity structures and improving seismic hazard assessment. However, standard methods such as finite difference or finite element are computationally…
Due to the emergency and homogenization of Artificial Intelligence (AI) technology development, transformer-based foundation models have revolutionized scientific applications, such as drug discovery, materials research, and astronomy.…
Estimating porosity models via seismic data is challenging due to the signal noise and insufficient resolution of seismic data. Although impedance inversion is often used by combining with well logs, several hurdles remain to retrieve…
I demonstrate that the conventional seismic full-waveform inversion algorithm can be constructed as a recurrent neural network and so implemented using deep learning software such as TensorFlow. Applying another deep learning concept, the…
In Transformer-based neural machine translation (NMT), the positional encoding mechanism helps the self-attention networks to learn the source representation with order dependency, which makes the Transformer-based NMT achieve…
Seismic waveform modeling is a powerful tool for determining earth structure models and unraveling earthquake rupture processes, but it is usually computationally expensive. We introduce a scheme to vastly accelerate these calculations with…
Incremental semantic segmentation(ISS) is an emerging task where old model is updated by incrementally adding new classes. At present, methods based on convolutional neural networks are dominant in ISS. However, studies have shown that such…
Newtonian machine learning (NML) is a wave-equation inversion method that inverts single-dimensional latent space (LS) features of the seismic data for retrieving the subsurface background velocity model. The single-dimensional LS features…
Seismic data noise processing is an important part of seismic exploration data processing, and the effect of noise elimination is directly related to the follow-up processing of data. In response to this problem, many authors have proposed…
We introduce a new way of learning to encode position information for non-recurrent models, such as Transformer models. Unlike RNN and LSTM, which contain inductive bias by loading the input tokens sequentially, non-recurrent models are…
We explore deep autoregressive Transformer models in language modeling for speech recognition. We focus on two aspects. First, we revisit Transformer model configurations specifically for language modeling. We show that well configured…
Since the superiority of Transformer in learning long-term dependency, the sign language Transformer model achieves remarkable progress in Sign Language Recognition (SLR) and Translation (SLT). However, there are several issues with the…
In this work, we study optimization methods that leverage the linear minimization oracle (LMO) over a norm-ball. We propose a new stochastic family of algorithms that uses the LMO to adapt to the geometry of the problem and, perhaps…
Real time, accurate passive seismic event detection is a critical safety measure across a range of monitoring applications from reservoir stability to carbon storage to volcanic tremor detection. The most common detection procedure remains…
Traffic forecasting is an indispensable part of Intelligent transportation systems (ITS), and long-term network-wide accurate traffic speed forecasting is one of the most challenging tasks. Recently, deep learning methods have become…