Related papers: Astronomical data organization, management and acc…
Scientific communities naturally tend to organize around data ecosystems created by the combination of their observational devices, their data repositories, and the workflows essential to carry their research from observation to discovery.…
Data lakes are becoming increasingly prevalent for big data management and data analytics. In contrast to traditional 'schema-on-write' approaches such as data warehouses, data lakes are repositories storing raw data in its original formats…
The start of data taking at the Large Hadron Collider will herald a new era in data volumes and distributed processing in particle physics. Data volumes of hundreds of Terabytes will be shipped to Tier-2 centres for analysis by the LHC…
The SKA will be capable of producing a stream of science data products that are Exa-scale in terms of their storage and processing requirements. This Google-scale enterprise is attracting considerable international interest and excitement…
With new emerging technologies, such as satellites and drones, archaeologists collect data over large areas. However, it becomes difficult to process such data in time. Archaeological data also have many different formats (images, texts,…
The Galactica simulation database is a platform designed to assist computational astrophysicists with their open science approach based on FAIR (Findable, Accessible, Interoperable, Reusable) principles. It offers the means to publish their…
Large High Energy Physics (HEP) experiments adopted a distributed computing model more than a decade ago. WLCG, the global computing infrastructure for LHC, in partnership with the US Open Science Grid, has achieved data management at the…
Scientists in all domains face a data avalanche - both from better instruments and from improved simulations. We believe that computer science tools and computer scientists are in a position to help all the sciences by building tools and…
Advanced instruments in a variety of scientific domains are collecting massive amounts of data that must be post-processed and organized to support scientific research activities. Astronomers have been pioneers in the use of databases to…
Data management can be a complex challenge in fields such as bioinformatics and health sciences, which continuously generate extensive heterogeneous datasets. In the context of collaborative global health initiatives, secure storage and…
The next-generation astronomy digital archives will cover most of the universe at fine resolution in many wave-lengths, from X-rays to ultraviolet, optical, and infrared. The archives will be stored at diverse geographical locations. One of…
In time-domain astronomy, STLF (Short-Timescale and Large Field-of-view) sky survey is the latest way of sky observation. Compared to traditional sky survey who can only find astronomical phenomena, STLF sky survey can even reveal how short…
As the amount of scientific data continues to grow at ever faster rates, the research community is increasingly in need of flexible computational infrastructure that can support the entirety of the data science lifecycle, including…
Astrophysics and Space Science are becoming increasingly characterised by what is now known as "big data", the bottlenecks for progress partly shifting from data acquisition to "data mining". Truth is that the amount and rate of data…
The Sloan Digital Sky Survey Science Archive is the first in a series of multi-Terabyte digital archives in Astronomy and other data-intensive sciences. To facilitate data mining in the SDSS archive, we adapted a commercial database engine…
With the growing number and increasing availability of shared-use instruments and observatories, observational data is becoming an essential part of application workflows and contributor to scientific discoveries in a range of disciplines.…
For more than a decade the Joint Astronomy Centre has been developing software tools to simplify observing and make it possible to use the telescopes in many different operational modes. In order to support remote operations the data…
The situation of data sharing in astronomy is positioned in the current general context of a political push towards, and rapid development of, scientific data sharing. Data is already one of the major infrastructures of astronomy, thanks to…
Nowadays, many scientific areas share the same broad requirements of being able to deal with massive and distributed datasets while, when possible, being integrated with services and applications. In order to solve the growing gap between…
The next-generation astronomy archives will cover most of the universe at fine resolution in many wavelengths. One of the first of these projects, the Sloan Digital Sky Survey (SDSS) will create a 5-wavelength catalog over 10,000 square…