Related papers: Data engineering for archive evolution
It is argued that the astronomy of the twenty-first century will be dominated by computer-based manipulation of huge homogeneous surveys of various types of astronomical objects. Furthermore combination of all observations with large…
Data science has emerged from the proliferation of digital data, coupled with advances in algorithms, software and hardware (e.g., GPU computing). Innovations in structural biology have been driven by similar factors, spurring us to ask:…
Astronomy looks after its data better than most disciplines, and it is no coincidence that the consensus standard for the archival preservation of all types of digital assets -- the OAIS Reference Model -- emerged originally from the space…
We explore how astronomers take observational data from telescopes, process them into usable scientific data products, curate them for later use, and reuse data for further inquiry. Astronomers have invested heavily in knowledge…
The advent of modern technology, permitting the measurement of thousands of characteristics simultaneously, has given rise to floods of data characterized by many large or even huge datasets. This new paradigm presents extraordinary…
Modern scientific repositories are growing rapidly in size. Scientists are increasingly interested in viewing the latest data as part of query results. Current scientific middleware cache systems, however, assume repositories are static.…
Astronomy has been at the forefront of the development of the techniques and methodologies of data intensive science for over a decade with large sky surveys and distributed efforts such as the Virtual Observatory. However, it faces a new…
The FITS file format has become the de facto standard for sharing, analyzing, and archiving astronomy data over the last four decades. FITS was adopted by astronomers in the early 1980s to overcome incompatibilities between operating…
Astronomical data generally consists of 2 or more high-resolution axes, e.g., X,Y position on the sky or wavelength and position-along-one-axis (long-slit spectrometer). Analyzing these multi-dimension observations requires combining 3D…
State of the art methods in astronomical image reconstruction rely on the resolution of a regularized or constrained optimization problem. Solving this problem can be computationally intensive and usually leads to a quadratic or at least…
Most domains of science are experiencing a paradigm shift due to the advent of a new generation of instruments and detectors which produce data and data streams at an unprecedented rate. The scientific exploitation of these data, namely…
The study explored the usage of astronomical observations for the identification and tracking of artificial satellites. Spacecraft streaks on astronomical images are a growing issue for the astronomical community. The increasing number of…
It is becoming common to archive research datasets that are not only large but also numerous. In addition, their corresponding metadata and the software required to analyse or display them need to be archived. Yet the manual curation of…
The Flexible Image Transport System (FITS) standard has been a great boon to astronomy, allowing observatories, scientists and the public to exchange astronomical information easily. The FITS standard is, however, showing its age. Developed…
Sky surveys represent a fundamental data basis for astronomy. We use them to map in a systematic way the universe and its constituents, and to discover new types of objects or phenomena. We review the subject, with an emphasis on the…
Advances in extragalactic astronomy have prompted the development of increasingly realistic models which aim to describe the formation and evolution of galaxies. We review the philosophy behind one such technique, called semi-analytic…
Information and data exchange is an important aspect of scientific progress. In computational materials science, a prerequisite for smooth data exchange is standardization, which means using agreed conventions for, e.g., units, zero base…
With new emerging technologies, such as satellites and drones, archaeologists collect data over large areas. However, it becomes difficult to process such data in time. Archaeological data also have many different formats (images, texts,…
The increase of astronomical data produced by a new generation of observational tools poses the need to distribute data and to bring computation close to the data. Trying to answer this need, we set up a federated data and computing…
The modelling of massive star evolution is a complex task, and is very sensitive to the way physical processes (such as convection, rotation, mass loss, etc.) are included in stellar evolution code. Moreover, the very high observed fraction…