Related papers: CANFAR+Skytree: A Cloud Computing and Data Mining …
A common task in scientific computing is the derivation of data. This workflow extracts the most important information from large input data and stores it in smaller derived data objects. The derived data objects can then be used for…
The new generation of high-resolution broad-band radio telescopes, like the Low Frequency Array (LOFAR), produces, depending on the level of compression, between 1 to 10 TB of data per hour after correlation. Such a large amount of…
Astrophysics lies at the crossroads of big datasets (such as the Large Synoptic Survey Telescope and Gaia), open source software to visualize and interpret high dimensional datasets (such as Glue, WorldWide Telescope, and OpenSpace), and…
Following the second HST servicing mission in 1997 when the STIS instrument was installed and the capability for parallel observations was enhanced, a substantial archive of non-proprietary parallel data has been accumulating. In this…
The connect-fill-run workflow paradigm, widely adopted in mature software engineering, accelerates collaborative development. However, computational chemistry, computational materials science, and computational biology face persistent…
The CASPER (Collaboration for Astronomy Signal Processing and Electronic Research) toolflow is a widely used framework for designing and implementing digital signal processing systems, particularly in the field of radio astronomy. It…
Data processing pipelines represent an important slice of the astronomical software library that include chains of processes that transform raw data into valuable information via data reduction and analysis. In this work we present Corral,…
Cross-matching operation, which is to find corresponding data for the same celestial object or region from multiple catalogues,is indispensable to astronomical data analysis and research. Due to the large amount of astronomical catalogues…
Time series data of celestial objects are commonly used to study valuable and unexpected objects such as extrasolar planets and supernova in time domain astronomy. Due to the rapid growth of data volume, traditional manual methods are…
We describe the application of data mining algorithms to research problems in astronomy. We posit that data mining has always been fundamental to astronomical research, since data mining is the basis of evidence-based discovery, including…
Traditional science searched for new objects and phenomena that led to discoveries. Tomorrow's science will combine together the large pool of information in scientific archives and make discoveries. Scienthists are currently keen to…
Cloud applications are increasingly distributing data across multiple regions and cloud providers. Unfortunately, wide-area bulk data transfers are often slow, bottlenecking applications. We demonstrate that it is possible to significantly…
Astronomy is experiencing a rapid growth in data size and complexity. This change fosters the development of data-driven science as a useful companion to the common model-driven data analysis paradigm, where astronomers develop automatic…
Datasets in the computer vision academic research community are primarily static. Once a dataset is accepted as a benchmark for a computer vision task, researchers working on this task will not alter it in order to make their results…
We describe preliminary investigations of using Docker for the deployment and testing of astronomy software. Docker is a relatively new containerisation technology that is developing rapidly and being adopted across a range of domains. It…
With the volume and availability of astronomical data growing rapidly, astronomers will soon rely on the use of machine learning algorithms in their daily work. This proceeding aims to give an overview of what machine learning is and delve…
The European Open Science Cloud (EOSC) aims to create a federated environment for hosting and processing research data to support science in all disciplines without geographical boundaries, such that data, software, methods and publications…
Modern scientific applications are increasingly decomposable into individual functions that may be deployed across distributed and diverse cyberinfrastructure such as supercomputers, clouds, and accelerators. Such applications call for new…
Cloud computing offers the potential to help scientists to process massive number of computing resources often required in machine learning application such as computer vision problems. This proposal would like to show that which benefits…
The application of machine learning (ML) methods to the analysis of astrophysical datasets is on the rise, particularly as the computing power and complex algorithms become more powerful and accessible. As the field of ML enjoys a…