Related papers: Transformers as Strong Lens Detectors- From Simula…

The observables in a strong gravitational lens are usually just the image positions and sometimes the flux ratios. We develop a new and simple algorithm which allows a set of models to be fitted exactly to the observations. Taking our cue…

From the nature of dark matter to the rate of expansion of our Universe, observations of distant galaxies distorted through strong gravitational lensing have the potential to answer some of the major open questions in astrophysics. Modeling…

Apart from the high accuracy of machine learning models, what interests many researchers in real-life problems (e.g., fraud detection, credit scoring) is to find hidden patterns in data; particularly when dealing with their challenging…

Transformer is a powerful architecture that achieves superior performance on various sequence learning tasks, including neural machine translation, language understanding, and sequence prediction. At the core of the Transformer is the…

Motivated by discovering strongly-lensed supernovae, gravitational waves, and kilonovae in the 2020s, we investigate whether to build a watch-list of clusters based on observed cluster properties (i.e. lens-plane selection) or on the…

Modeling the mass distribution of galaxy-scale strong gravitational lenses is a task of increasing difficulty. The high-resolution and depth of imaging data now available render simple analytical forms ineffective at capturing lens…

In this paper, we introduce the prior knowledge, multi-scale structure, into self-attention modules. We propose a Multi-Scale Transformer which uses multi-scale multi-head self-attention to capture features from different scales. Based on…

Modern machine learning models for computer vision exceed humans in accuracy on specific visual recognition tasks, notably on datasets like ImageNet. However, high accuracy can be achieved in many ways. The particular decision function…

We present a new gravitational lens modelling technique designed to model high-resolution interferometric observations with large numbers of visibilities without the need to pre-average the data in time or frequency. We demonstrate the…

The Transformer is an extremely powerful and prominent deep learning architecture. In this work, we challenge the commonly held belief in deep learning that going deeper is better, and show an alternative design approach that is building…

The study of strong-lensing systems conventionally involves constructing a mass distribution that can reproduce the observed multiply-imaging properties. Such mass reconstructions are generically non-unique. Here, we present an alternative…

Like many areas of astrophysics and cosmology, the Vera C. Rubin Observatory will be transformational for almost all the applications of strong lensing, thanks to the dramatic increase in the number of known strong lenses by two orders of…

The process by which Large Language Models (LLMs) acquire complex capabilities during training remains a key open question in mechanistic interpretability. This project investigates whether these learning dynamics can be characterized…

Deep learning has profoundly impacted domains such as computer vision and natural language processing by uncovering complex patterns in vast datasets. However, the reliance on extensive labeled data poses significant challenges, including…

Transformers have achieved success in both language and vision domains. However, it is prohibitively expensive to scale them to long sequences such as long documents or high-resolution images, because self-attention mechanism has quadratic…

The attention operator is arguably the key distinguishing factor of transformer architectures, which have demonstrated state-of-the-art performance on a variety of tasks. However, transformer attention operators often impose a significant…

Recent advances in transformer-based Large Language Models (LLMs) have demonstrated remarkable capabilities across various tasks. However, their quadratic computational complexity concerning sequence length remains a significant bottleneck…

With the advent of new spectroscopic surveys from ground and space, observing up to hundreds of millions of galaxies, spectra classification will become overwhelming for standard analysis techniques. To prepare for this challenge, we…

Typically, binary classification lens-finding schemes are used to discriminate between lens candidates and non-lenses. However, these models often suffer from substantial false-positive classifications. Such false positives frequently occur…

Unconstrained remote gaze tracking using off-the-shelf cameras is a challenging problem. Recently, promising algorithms for appearance-based gaze estimation using convolutional neural networks (CNN) have been proposed. Improving their…