相关论文: Scientific Computing Meets Big Data Technology: An…
Scientific results in high-energy physics and in many other fields often rely on complex software stacks. In order to support reproducibility and scrutiny of the results, it is good practice to use open source software and to cite software…
Astronomy is entering in a new era of Extreme Intensive Data Computation and we have identified three major issues the new generation of projects have to face: Resource optimization, Heterogeneous Software Ecosystem and Data Transfer. We…
We explore how astronomers take observational data from telescopes, process them into usable scientific data products, curate them for later use, and reuse data for further inquiry. Astronomers have invested heavily in knowledge…
Microscopy, in particular scanning probe and electron microscopy, has been pivotal in improving our understanding of structure-function relationships at the nanoscale and is by now ubiquitous in most research characterization labs and…
Experimental Particle Physics has been at the forefront of analyzing the worlds largest datasets for decades. The HEP community was the first to develop suitable software and computing tools for this task. In recent times, new toolkits and…
Modern scientific applications predominantly run on large-scale computing platforms, necessitating collaboration between scientific domain experts and high-performance computing (HPC) experts. While domain experts are often skilled in…
Planetary science space missions need high quality software ed efficient algorithms in order to extract innovative scientific results from flight data. Reliable and efficient software technologies are increasingly vital to improve and…
Apache Flink is an open-source system for scalable processing of batch and streaming data. Flink does not natively support efficient processing of spatial data streams, which is a requirement of many applications dealing with spatial data.…
An ESO internal ALMA development study, BRAIN, is addressing the ill-posed inverse problem of synthesis image analysis employing astrostatistics and astroinformatics. These emerging fields of research offer interdisciplinary approaches at…
In the era of data-driven science, conducting computational experiments that involve analysing large datasets using heterogeneous computational clusters, is part of the everyday routine for many scientists. Moreover, to ensure the…
Critical goals of scientific computing are to increase scientific rigor, reproducibility, and transparency while keeping up with ever-increasing computational demands. This work presents an integrated framework well-suited for data…
High-performance scientific applications require more and more compute power. The concurrent use of multiple distributed compute resources is vital for making scientific progress. The resulting distributed system, a so-called Jungle…
The cycle of scientific discovery is frequently bottlenecked by the slow, manual creation of software to support computational experiments\cite{hannay2009how}. To address this, we present Empirical Research Assistance (ERA), an AI system…
Background. Life science is increasingly driven by Big Data analytics, and the MapReduce programming model has been proven successful for data-intensive analyses. However, current MapReduce frameworks offer poor support for reusing existing…
Software is a critical aspect of large-scale science, providing essential capabilities for making scientific discoveries. Large-scale scientific projects are vast in scope, with lifespans measured in decades and costs exceeding hundreds of…
Although Cloud computing emerged for business applications in industry, public Cloud services have been widely accepted and encouraged for scientific computing in academia. The recently available Google Compute Engine (GCE) is claimed to…
We review some aspects of the current state of data-intensive astronomy, its methods, and some outstanding data analysis challenges. Astronomy is at the forefront of "big data" science, with exponentially growing data volumes and data…
The increasing availability of high-quality optical and near-infrared spectroscopic data, as well as advances in modelling techniques, have greatly expanded the scientific potential of spectroscopic studies. However, the software tools…
Developing sustainable software for the scientific community requires expertise in software engineering and domain science. This can be challenging due to the unique needs of scientific software, the insufficient resources for software…
In the era of data explosion, a growing number of data-intensive computing frameworks, such as Apache Hadoop and Spark, have been proposed to handle the massive volume of unstructured data in parallel. Since programming models provided by…