Related papers: Selective State-Space Models for Koopman-based Dat…
Transformers have revolutionized deep learning across various tasks, including audio representation learning, due to their powerful modeling capabilities. However, they often suffer from quadratic complexity in both GPU memory usage and…
Existing Transformer-based models for point cloud analysis suffer from quadratic complexity, leading to compromised point cloud resolution and information loss. In contrast, the newly proposed Mamba model, based on state space models (SSM),…
This letter proposes a novel and highly efficient distribution system state estimation (DSSE) algorithm with nonlinear measurements from supervisory control and data acquisition (SCADA) systems. Conventional DSSE, i.e., a weighted least…
State Space Models (SSMs) have emerged as a promising alternative to the popular transformer-based models and have been increasingly gaining attention. Compared to transformers, SSMs excel at tasks with sequential data or longer contexts,…
Today's power generation and distribution networks are quickly moving toward automated control and integration of renewable resources - a complex, integrated system termed the Smart Grid. A key component in planning and managing of Smart…
Distribution system state estimation (DSSE) is paramount for effective state monitoring and control. However, stochastic outputs of renewables and asynchronous streaming of multi-rate measurements in practical systems largely degrade the…
Semantic segmentation is commonly used for Oil Spill Detection (OSD) in remote sensing images. However, the limited availability of labelled oil spill samples and class imbalance present significant challenges that can reduce detection…
Hybrid AC/DC distribution systems are becoming a popular means to accommodate the increasing penetration of distributed energy resources and flexible loads. This paper proposes a distributed and robust state estimation (DRSE) method for…
Token-free language models learn directly from raw bytes and remove the inductive bias of subword tokenization. Operating on bytes, however, results in significantly longer sequences. In this setting, standard autoregressive Transformers…
Dynamic state estimation (DSE) is vital in modern power systems with numerous inverter-based distributed energy resources including solar and wind, ensuring real-time accuracy for tracking system variables and optimizing grid stability.…
Recent works have demonstrated that attention-based transformer and large language model (LLM) architectures can achieve strong channel state prediction (CSP) performance by capturing long-range temporal dependencies across channel state…
Due to the intensifying impacts of extreme climate changes, drought forecasting (DF), which aims to predict droughts from historical meteorological data, has become increasingly critical for monitoring and managing water resources. Though…
A distribution system can flexibly adjust its substation-level power output by aggregating its local distributed energy resources (DERs). Due to DER and network constraints, characterizing the exact feasible power output region is…
Spatial-Temporal Graph (STG) data is characterized as dynamic, heterogenous, and non-stationary, leading to the continuous challenge of spatial-temporal graph learning. In the past few years, various GNN-based methods have been proposed to…
State Space Models (SSMs)-most notably RNNs-have historically played a central role in sequential modeling. Although attention mechanisms such as Transformers have since dominated due to their ability to model global context, their…
Distribution systems of the future smart grid require enhancements to the reliability of distribution system state estimation (DSSE) in the face of low measurement redundancy, unsynchronized measurements, and dynamic load profiles. Micro…
State-space models (SSMs) have recently demonstrated competitive performance to transformers at large-scale language modeling benchmarks while achieving linear time and memory complexity as a function of sequence length. Mamba, a recently…
In recent years, Transformers have become the de-facto architecture for long-term sequence forecasting (LTSF), but faces challenges such as quadratic complexity and permutation invariant bias. A recent model, Mamba, based on selective state…
Dynamic structural equation models (DSEMs) combine time-series modeling of within-person processes with hierarchical modeling of between-person differences and differences between timepoints, and have become very popular for the analysis of…
Large pre-trained models have achieved outstanding results in sequence modeling. The Transformer block and its attention mechanism have been the main drivers of the success of these models. Recently, alternative architectures, such as…