Related papers: Transformers as Strong Lens Detectors- From Simula…

A few models have tried to tackle the link prediction problem, also known as knowledge graph completion, by embedding knowledge graphs in comparably lower dimensions. However, the state-of-the-art results are attained at the cost of…

In this paper we investigate transformer architectures designed for partially observable online reinforcement learning. The self-attention mechanism in the transformer architecture is capable of capturing long-range dependencies and it is…

A total of 10 good candidates for gravitational lensing have been discovered in the WFPC2 images from the HST Medium Deep Survey (MDS) and archival primary observations. These candidate lenses are unique HST discoveries, i.e. they are faint…

Line detection is a basic digital image processing operation used by higher-level processing methods. Recently, transformer-based methods for line detection have proven to be more accurate than methods based on CNNs, at the expense of…

State-of-the-art results on neural machine translation often use attentional sequence-to-sequence models with some form of convolution or recursion. Vaswani et al. (2017) propose a new architecture that avoids recurrence and convolution…

Transformers have had tremendous impact for several sequence related tasks, largely due to their ability to retrieve from any part of the sequence via softmax based dot-product attention. This mechanism plays a crucial role in Transformer's…

Transformer models have achieved remarkable results in a wide range of applications. However, their scalability is hampered by the quadratic time and memory complexity of the self-attention mechanism concerning the sequence length. This…

Tens of thousands of galaxy-galaxy strong lensing systems are expected to be discovered by the end of the decade. These will form a vast new dataset that can be used to probe subgalactic dark matter structures through its gravitational…

Representation learning has significantly been developed with the advance of contrastive learning methods. Most of those methods have benefited from various data augmentations that are carefully designated to maintain their identities so…

We present a systematic search for wide-separation (Einstein radius >1.5"), galaxy-scale strong lenses in the 30 000 sq.deg of the Pan-STARRS 3pi survey on the Northern sky. With long time delays of a few days to weeks, such systems are…

Transformers have been successfully used in various fields and are becoming the standard tools in computer vision. However, self-attention, a core component of transformers, has a quadratic complexity problem, which limits the use of…

Recent advances in Transformer architectures have empowered their empirical success in a variety of tasks across different domains. However, existing works mainly focus on predictive accuracy and computational cost, without considering…

While Large Language Models (LLMs) achieve strong performance across diverse tasks, their inference dynamics remain poorly understood because of the limited resolution of existing analysis tools. In this work, we identify an intrinsic…

Vision Transformers have been rapidly uprising in computer vision thanks to their outstanding scaling trends, and gradually replacing convolutional neural networks (CNNs). Recent works on self-supervised learning (SSL) introduce siamese…

The past decades have witnessed a lot of progress in gravitational lensing with two main targets: stars and galaxies (with active galactic nuclei). The success is partially attributed to the continuous luminescence of these sources making…

Strong gravitationally lensed supernovae (glSNe) are a powerful probe to obtain a measure of the expansion rate of the Universe, but they are also extremely rare. To date, only two glSNe with multiple images strongly lensed by galaxies have…

Recently, CNN object detectors have achieved high accuracy on remote sensing images but require huge labor and time costs on annotation. In this paper, we propose a new uncertainty-based active learning which can select images with more…

Galaxy-scale strong gravitational lenses are valuable objects for a variety of astrophysical and cosmological applications. Strong lensing galaxies are rare, so efficient search methods, such as convolutional neural networks, are often used…

This is a follow-up sensitivity study on r-mode gravitational wave signals from newborn neutron stars illustrating the applicability of machine learning algorithms for the detection of long-lived gravitational-wave transients. In this…

Large-scale vision foundation models have made significant progress in visual tasks on natural images, with vision transformers being the primary choice due to their good scalability and representation ability. However, large-scale models…