Related papers: Data engineering for archive evolution
We briefly review some constraints (Owing to the limited number of pages of present review, only a sub-sample of the topics discussed during the talk are briefly summarized. For the interested readers we are pleased to send them upon…
Current extragalactic databases are reviewed, including object-oriented databases, astronomical catalogues and compilations, as well as image archives and object catalogues from large-scale surveys. One challenge of the future will be to…
We present a summary of the major contributions to the Special Session on Data Management held at the IAU General Assembly in Prague in 2006. While recent years have seen enormous improvements in access to astronomical data, and the Virtual…
Fundamental changes are taking place in the way we do astronomy. In twenty years time, it is likely that most astronomers will never go near a cutting-edge telescope, which will be much more efficiently operated in service mode. They will…
Most scientific data will never be directly examined by scientists; rather it will be put into online databases where it will be analyzed and summarized by computer programs. Scientists increasingly see their instruments through online…
Astronomical observations already produce vast amounts of data through a new generation of telescopes that cannot be analyzed manually. Next-generation telescopes such as the Large Synoptic Survey Telescope and the Square Kilometer Array…
Astronomy is entering a new era as multiple, large area, digital sky surveys are in production. The resulting datasets are truly remarkable in their own right; however, a revolutionary step arises in the aggregation of complimentary…
Policy Brief on "Global Data in Astronomy: Challenges and Opportunities", distilled from the corresponding panel that was part of the discussions during S20 Policy Webinar on Astroinformatics for Sustainable Development held on 6-7 July…
Most major scientific results produced by ground-based gamma-ray telescopes in the last 30 years have been obtained by expert members of the collaborations operating these instruments. This is due to the proprietary data and software…
Over the past century, major advances in astronomy and astrophysics have been largely driven by improvements in instrumentation and data collection. With the amassing of high quality data from new telescopes, and especially with the advent…
In this work, we identify elements of effective machine learning datasets in astronomy and present suggestions for their design and creation. Machine learning has become an increasingly important tool for analyzing and understanding the…
The data volumes stored in telescope archives is constantly increasing due to the development and improvements in the instrumentation. Often the archives need to be stored over a distributed storage architecture, provided by independent…
Astrophysics has become a domain extremely rich of scientific data. Data mining tools are needed for information extraction from such large datasets. This asks for an approach to data management emphasizing the efficiency and simplicity of…
Many astronomy data centres still work on filesystems. Industry has moved on; current practice in computing infrastructure is to achieve Big Data scalability using object stores rather than POSIX file systems. This presents us with…
Massive stars condition the evolution of the interstellar medium by the amount of energy released during their lives and especially by their deaths as supernova explosions. The vast amounts of spectroscopic data for massive stars provided…
As a scientific discipline, Astronomy is rather unique. We only have one laboratory, the Universe, and we cannot, of course, change the initial conditions and study the resulting effects. On top of this, acquiring Astronomical data has…
The data warehouse (DW) technology was developed to integrate heterogeneous information sources for analysis purposes. Information sources are more and more autonomous and they often change their content due to perpetual transactions (data…
Science is becoming very data intensive1. Today's astronomy datasets with tens of millions of galaxies already present substantial challenges for data mining. In less than 10 years the catalogs are expected to grow to billions of objects,…
In this review, we explore the historical development and future prospects of artificial intelligence (AI) and deep learning in astronomy. We trace the evolution of connectionism in astronomy through its three waves, from the early use of…
We should be in a golden age of scientific discovery, given that we have more data and more compute power available than ever before, plus a new generation of algorithms that can learn effectively from data. But paradoxically, in many…