Related papers: OpenCluster: A Flexible Distributed Computing Fram…
For more than a decade the Joint Astronomy Centre has been developing software tools to simplify observing and make it possible to use the telescopes in many different operational modes. In order to support remote operations the data…
The Chinese Spectral RadioHeliograph (CSRH) is a synthetic aperture radio interferometer built in Inner Mongolia, China. As a solar-dedicated interferometric array, CSRH is capable of producing high quality radio images at frequency range…
Spectral clustering is a popular method for effectively clustering nonlinearly separable data. However, computational limitations, memory requirements, and the inability to perform incremental learning challenge its widespread application.…
Where appropriate repositories are not available to support all relevant astronomical data products, data can fall into darkness: unseen and unavailable for future reference and re-use. Some data in this category are legacy or old data, but…
Modern galaxy cluster science is a multi-wavelength endeavor with cornerstones provided by X-ray, optical/IR, mm, and radio measurements. In combination, these observations enable the construction of large, clean, complete cluster catalogs,…
Open Universe for blazars is a set of high-transparency data products for blazar science, and the tools designed to generate them. Blazar astrophysics is becoming increasingly data driven, depending on the integration and combined analysis…
This paper describes by example how astronomers can use cloud-computing resources offered by Amazon Web Services (AWS) to create new datasets at scale. We have created from existing surveys an atlas of the Galactic Plane at 16 wavelengths…
Data processing pipelines represent an important slice of the astronomical software library that include chains of processes that transform raw data into valuable information via data reduction and analysis. In this work we present Corral,…
Current catalogues of open clusters are rather heterogeneous and incomplete lists of clusters than true catalogues. Before there has been no attempts of automatic search for open clusters in huge photometric catalogues using homogeneous…
The pervasive adoption of Deep Learning (DL) and Graph Processing (GP) makes it a de facto requirement to build large-scale clusters of heterogeneous accelerators including GPUs and FPGAs. The OpenCL programming framework can be used on the…
Automated and computerised control of scientific instrumentation is almost ubiquitous in the modern laboratory. Most instrumentation is controlled over decades old communication busses or is accessed via proprietary system libraries. This…
A major challenge in modern radio astronomy is dealing with the massive data volumes generated by wide-bandwidth receivers. Such massive data rates are often too great for a single device to cope, and so processing must be split across…
DSPSR is a high-performance, open-source, object-oriented, digital signal processing software library and application suite for use in radio pulsar astronomy. Written primarily in C++, the library implements an extensive range of modular…
We review some aspects of the current state of data-intensive astronomy, its methods, and some outstanding data analysis challenges. Astronomy is at the forefront of "big data" science, with exponentially growing data volumes and data…
We describe a new open source package for calculating properties of galaxy clusters, including NFW halo profiles with and without the effects of cluster miscentering. This pure-Python package, cluster-lensing, provides well-documented and…
Astrophysics has become a domain extremely rich of scientific data. Data mining tools are needed for information extraction from such large datasets. This asks for an approach to data management emphasizing the efficiency and simplicity of…
Apache Spark is a Big Data framework for working on large distributed datasets. Although widely used in the industry, it remains rather limited in the academic community or often restricted to software engineers. The goal of this paper is…
Spectral clustering requires the time-consuming decomposition of the Laplacian matrix of the similarity graph, thus limiting its applicability to large datasets. To improve the efficiency of spectral clustering, a top-down approach was…
Many organizations routinely analyze large datasets using systems for distributed data-parallel processing and clusters of commodity resources. Yet, users need to configure adequate resources for their data processing jobs. This requires…
Access to astronomical data through archives and VO is essential but does not solve all problems. Availability of appropriate software for analyzing the data is often equally important for the efficiency with which a researcher can publish…