Related papers: Astro-WISE: Chaining to the Universe
Interactive visualization of astronomical catalogs requires novel techniques due to the huge volumes and complex structure of the data produced by existing and upcoming astronomical surveys. The creation as well as the disclosure of the…
The massive data sets from today's particle physics experiments present a variety of challenges amenable to the tools developed by the statistics community. From the real-time decision of what subset of data to record on permanent storage,…
Observational astronomy has changed drastically in the last decade: manually driven target-by-target instruments have been replaced by fully automated robotic telescopes. Data acquisition methods have advanced to the point that terabytes of…
The Target infrastructure has been specially built as a storage and compute infrastructure for the information systems derived from Astro-WISE. This infrastructure will be used by several applications that collaborate in the area of…
The ominous warnings of a `data deluge' in the life sciences from high-throughput DNA sequencing data are being supplanted by a second deluge, of cliches bemoaning our collective scientific fate unless we address the genomic data `tsunami'.…
Open Universe for blazars is a set of high-transparency data products for blazar science, and the tools designed to generate them. Blazar astrophysics is becoming increasingly data driven, depending on the integration and combined analysis…
Astronomy has always been at the forefront of information technology, moving from the era of photographic plates, to digital snapshots and now to digital movies of the sky. This has brought about a data explosion with multi- terabyte…
Over the past century, major advances in astronomy and astrophysics have been largely driven by improvements in instrumentation and data collection. With the amassing of high quality data from new telescopes, and especially with the advent…
Data-centric materials science is changing how materials are discovered, optimized, manufactured, and qualified, yet many deployment-limiting materials problems still depend on experimental, processing-rich, device-level, and field-relevant…
Earth observation analytics have the potential to transform many sectors. However, due to limited ground connections, it currently takes hours to days to download and analyze Earth observation data, diminishing the value of data for…
Cyber-physical systems increasingly rely on distributed computing platforms where sensing, computing, actuation, and communication resources are shared by a multitude of applications. Such `cyber-physical cloud computing platforms' present…
BEANS software is a web based, easy to install and maintain, new tool to store and analyse data in a distributed way for a massive amount of data. It provides a clear interface for querying, filtering, aggregating, and plotting data from an…
The promise of "free and open" multi-terabyte datasets often collides with harsh realities. While these datasets may be technically accessible, practical barriers -- from processing complexity to hidden costs -- create a system that…
Modern soft X-ray observatories can yield unique insights into time domain astrophysics, and a huge amount of information is stored - and largely unexploited - in data archives. Like a treasure-hunt, the EXTraS project harvested the…
This is a thought piece on data-intensive science requirements for databases and science centers. It argues that peta-scale datasets will be housed by science centers that provide substantial storage and processing for scientists who access…
The first step in a science project is the acquisition and understanding of the relevant data. This paper outlines the results of a project to design and test network tools specifically oriented at retrieving astronomical data. The tools…
Collaborations in astronomy and astrophysics are faced with numerous cyber infrastructure challenges, such as large data sets, the need to combine heterogeneous data sets, and the challenge to effectively collaborate on those large,…
Widespread e-commerce activity on the Internet has led to new opportunities to collect vast amounts of micro-level market and nonmarket data. In this paper we share our experiences in collecting, validating, storing and analyzing large…
The increasing volumes of data produced by high-throughput instruments coupled with advanced computational infrastructures for scientific computing have enabled what is often called a {\em Fourth Paradigm} for scientific research based on…
The emergence of a complex, large-scale organisation of cosmic matter into the Cosmic Web is a beautiful exemplification of how complexity can be produced by simple initial conditions and simple physical laws. In the epoch of Big Data in…