Related papers: An Hierarchical Approach to Big Data

Scientific exploitation of the ever increasing volumes of astronomical data requires efficient and practical methods for data access, visualisation, and analysis. Hierarchical sky tessellation techniques enable a multi-resolution approach…

This document presents HiPS, a hierarchical scheme for the description, storage and access of sky survey data. The system is based on hierarchical tiling of sky regions at finer and finer spatial resolution which facilitates a progressive…

Context. In the current era of multi-wavelength and multi-messenger astronomy, international organisations are actively working on the definition of new standards for the publication of astronomical data, and substantial effort is devoted…

HEALPix -- the Hierarchical Equal Area iso-Latitude Pixelization -- is a versatile data structure with an associated library of computational algorithms and visualization software that supports fast scientific applications executable…

The WorldWide Telescope(WWT) is a scientific visualization platform which can browse deep space images, star catalogs, and planetary remote sensing data from different observation facilities in a three-dimensional virtual scene. First…

HEALPix -- the Hierarchical Equal Area isoLatitude Pixelization -- has become a standard in high-energy and gravitational wave astronomy. Originally developed to improve the efficiency of all-sky Fourier analyses, it is now also utilized to…

In various astronomical projects it is crucial to have coordinates indexed tables. All sky optical and IR catalogues have up to 1 billion objects that will increase with forthcoming projects. Also partial sky surveys at various wavelengths…

We report here on a project that has developed a practical approach to processing all-sky image collections on cloud platforms, using as an exemplar application the creation of three-color Hierarchical Progressive Survey (HiPS) maps of the…

HEALPix is a Hierarchical, Equal Area, and iso-Latitude Pixelisation of the sphere designed to support efficiently - local operations on the pixel set, - a hierarchical tree structure for multi-resolution applications, and - the global Fast…

We describe a case study to use the Montage image mosaic engine to create maps of the ALLWISE image data set in the Hierarchical Progressive Survey (HiPS) sky-tesselation scheme. Our approach demonstrates that Montage reveals the science…

The rapid growth of imaging and spectroscopic surveys has intensified the need for efficient tools that support visual inspection, a practice that remains essential for tasks such as classification, catalog refinement, and validation of…

The amount of remote sensing (RS) data has increased at an unexpected scale, due to the rapid progress of earth-observation and the growth of satellite RS and sensor technologies. Traditional relational databases attend their limit to meet…

Data exploration and visualization systems are of great importance in the Big Data era, in which the volume and heterogeneity of available information make it difficult for humans to manually explore and analyse data. Most traditional…

Spatial data fusion is a bottleneck when it meets the scale of 10 billion records. Cross-matching celestial catalogs is just one example of this. To challenge this, we present a framework that enables efficient cross-matching using Learned…

The HI Parkes All-Sky Survey (HIPASS) Catalogue forms the largest uniform catalogue of HI sources compiled to date, with 4,315 sources identified purely by their HI content. The catalogue data comprise the southern region declination <+2…

The rapid growth of solar data is driving changes in the typical workflow and algorithmic approach to solar data analysis. We present recently deployed tools to aid this evolution and layout the path for future development. The majority of…

We review some aspects of the current state of data-intensive astronomy, its methods, and some outstanding data analysis challenges. Astronomy is at the forefront of "big data" science, with exponentially growing data volumes and data…

Data volumes from multiple sky surveys have grown from gigabytes into terabytes during the past decade, and will grow from terabytes into tens (or hundreds) of petabytes in the next decade. This exponential growth of new data both enables…

Astronomical researchers often think of analysis and visualization as separate tasks. In the case of high-dimensional data sets, though, interactive exploratory data visualization can give far more insight than an approach where data…

Heliophysics image data largely relies on a forty-year-old ecosystem built on the venerable Flexible Image Transport System (FITS) data standard. While many in situ measurements use newer standards, they are difficult to integrate with…