Related papers: The Astro-WISE Optical Image Pipeline: Development…
Radio astronomy observatories with high throughput back end instruments require real-time data processing. While computing hardware continues to advance rapidly, development of real-time processing pipelines remains difficult and…
Modern telescopes generate increasingly large and diverse datasets, often consisting of complex and morphologically rich structures. To efficiently explore such data requires automated methods that can extract and organize physically…
At the Canadian Astronomy Data Centre, we have combined our cloud computing system, CANFAR, with the world's most advanced machine learning software, Skytree, to create the world's first cloud computing system for data mining in astronomy.…
This article presents a newly developed Web portal called VisIVOWeb that aims to provide the astrophysical community with powerful visualization tools for large-scale data sets in the context of Web 2.0. VisIVOWeb can effectively handle…
This work explores the use of big data technologies deployed in the cloud for processing of astronomical data. We have applied Hadoop and Spark to the task of co-adding astronomical images. We compared the overhead and execution time of…
Digitized histopathology glass slides, known as Whole Slide Images (WSIs), are often several gigapixels large and contain sensitive metadata information, which makes distributed processing unfeasible. Moreover, artifacts in WSIs may result…
Petabyte-scale data volumes are generated by observations and simulations in modern astronomy and astrophysics. Storage, access, and data analysis are significantly hampered by such data volumes and are leading to the development of a new…
WEAVE, the new wide-field, massively multiplexed spectroscopic survey facility for the William Herschel Telescope, will see first light in late 2022. WEAVE comprises a new 2-degree field-of-view prime-focus corrector system, a nearly…
The volume of data generated by modern astronomical telescopes is extremely large and rapidly growing. However, current high-performance data processing architectures/frameworks are not well suited for astronomers because of their…
We describe a new polarized imaging pipeline implemented in the FHD software package. The pipeline is based on the optimal mapmaking imaging approach and performs horizon-to-horizon image reconstruction in all polarization modes. We discuss…
Current and upcoming radio-interferometers are expected to produce volumes of data of increasing size that need to be processed in order to generate the corresponding sky brightness distributions through imaging. This represents an…
We are developing automated systems to provide homogeneous calibration meta-data for heterogeneous imaging data, using the pixel content of the image alone where necessary. Standardized and complete calibration meta-data permit generative…
Earth observation analytics have the potential to transform many sectors. However, due to limited ground connections, it currently takes hours to days to download and analyze Earth observation data, diminishing the value of data for…
The all sky surveys done by the Palomar Observatory Schmidt, the European Southern Observatory Schmidt, and the United Kingdom Schmidt, the InfraRed Astronomical Satellite and the 2 Micron All Sky Survey have proven to be extremely useful…
The Herschel Common Science System (HCSS) (Ott et al. 2006) (Ott & Science Ground Segment Consortium 2010) is a substantial Java software package, accompanying the development of the Herschel Mission (Pilbratt et al. 2010), supporting all…
We present the data processing pipeline to generate calibrated data products from the Spectral and Photometric Imaging Receiver (SPIRE) imaging Fourier Transform Spectrometer on the Herschel Space Observatory. The pipeline processes…
(Scanning) transmission electron microscopy ((S)TEM) has significantly advanced materials science but faces challenges in correlating precise atomic structure information with the functional properties of devices due to its time-intensive…
Very High Resolution satellite and aerial imagery are used to monitor and conduct large scale surveys of ecological systems. Convolutional Neural Networks have successfully been employed to analyze such imagery to detect large animals and…
We present full-sky coadded maps created by uniformly combining the first five years of Wide-field Infrared Survey Explorer (WISE) and NEOWISE imaging at 3.4 microns (W1) and 4.6 microns (W2). By incorporating both pre-hibernation WISE…
Modern edge devices, such as cameras, drones, and Internet-of-Things nodes, rely on deep learning to enable a wide range of intelligent applications, including object recognition, environment perception, and autonomous navigation. However,…