相关论文: Integrated electro-optic attention nonlinearities …
Lithium niobate (LN) is an excellent nonlinear optical and electro-optic material that has found many applications in classical nonlinear optics, optical fiber communications and quantum photonics. Here we review the recent development of…
Various linear complexity models, such as Linear Transformer (LinFormer), State Space Model (SSM), and Linear RNN (LinRNN), have been proposed to replace the conventional softmax attention in Transformer structures. However, the optimal…
We demonstrated broadband highly efficient optical nonlinear processes in on-chip integrated lithium niobate (LN) microdisk resonators. The Q factors of the micro-resonators fabricated by femtosecond laser writing and chemo-mechanical…
Thin-film lithium niobate (TFLN) in the form of x- or z-cut lithium-niobate-on-insulator (LNOI) has recently popped up as a very promising and novel platform for developing integrated optoelectronic (nano)devices and exploring fundamental…
Transformer-based deep learning models are increasingly deployed on energy, and DRAM bandwidth constrained devices such as laptops and gaming consoles, which presents significant challenges in meeting the latency requirements of the models.…
Tunable nonlinearity facilitates the creation of reconfigurable nonlinear metasurfaces, enabling innovative applications in signal processing, light switching, and sensing. This paper presents a novel approach to electrically modulate SHG…
In this paper, we introduce a new nonlinear optical channel equalizer based on Transformers. By leveraging parallel computation and attending directly to the memory across a sequence of symbols, we show that Transformers can be used…
Linear attention significantly reduces the computational complexity of Transformers from quadratic to linear, yet it consistently lags behind softmax-based attention in performance. We identify the root cause of this degradation as the…
Thin-film lithium niobate (TFLN) based frequency doublers have been widely recognized as essential components for both classical and quantum optical communications. Nonetheless, the efficiency of these devices is hindered by imperfections…
Vision-and-language navigation (VLN), a frontier study aiming to pave the way for general-purpose robots, has been a hot topic in the computer vision and natural language processing community. The VLN task requires an agent to navigate to a…
Softmax self-attention often assigns disproportionate weight to semantically uninformative tokens such as special tokens and punctuation, a phenomenon known as attention noise. While recent methods like Cog Attention and the Differential…
Magnetotelluric (MT) forward modeling is fundamental for improving the accuracy and efficiency of MT inversion. Neural operators (NOs) have been effectively used for rapid MT forward modeling, demonstrating their promising performance in…
Thin-film lithium niobate is a promising platform owing to its large electro-optic coefficients and low propagation loss. However, the large footprints of devices limit their application in large-scale integrated optical systems. A crucial…
Vision Transformers (ViTs) that leverage self-attention mechanism have shown superior performance on many classical vision tasks compared to convolutional neural networks (CNNs) and gain increasing popularity recently. Existing ViTs works…
Thin-film lithium niobate (TFLN) is an emerging platform for compact, low-power nonlinear-optical devices, and has been used extensively for near-infrared frequency conversion. Recent work has extended these devices to mid-infrared…
While linear attention reduces the quadratic complexity of standard Transformers to linear time, it often lags behind in expressivity due to the removal of softmax normalization. This omission eliminates \emph{global competition}, a…
Recent advances in multimodal large language models (MLLMs) have enabled impressive progress in vision-language understanding, yet their high computational cost limits deployment in resource-constrained scenarios such as personal…
In display technologies or data processing, planar and subwavelength free-space components suited for flat photonic devices are needed. Metasurfaces, which shape the optical wavefront within hundreds of nanometers, can provide a solution…
Thin-film lithium niobate (TFLN) has recently emerged as a promising platform for integrated nonlinear photonics, enabling the use of optical parametric amplifiers (OPAs) for applications in quantum information processing, precision…
Electro-optic modulators are utilized ubiquitously ranging from applications in data communication to photonic neural networks. While tremendous progress has been made over the years, efficient phase-shifting modulators are challenged with…