Related papers: Development of a High Throughput Cloud-Based Data …
Cloud providers typically charge for their services. There are diverse pricing models which often follow a pay-per-use paradigm. The consumers' payments are expected to cover all cost which incurs to the provider for processing, storage,…
Applications that fuse machine learning and simulation can benefit from the use of multiple computing resources, with, for example, simulation codes running on highly parallel supercomputers and AI training and inference tasks on…
We describe the design and implementation of a high performance cloud that we have used to archive, analyze and mine large distributed data sets. By a cloud, we mean an infrastructure that provides resources and/or services over the…
Currently, the processing of scientific data in astroparticle physics is based on various distributed technologies, the most common of which are Grid and cloud computing. The most frequently discussed approaches are focused on large and…
The increasing adoption of low-cost environmental sensors and AI-enabled applications has accelerated the demand for scalable and resilient data infrastructures, particularly in data-scarce and resource-constrained regions. This paper…
Curating, processing, and combining large-scale medical imaging datasets from national studies is a non-trivial task due to the intense computation and data throughput required, variability of acquired data, and associated financial…
Unsupervised machine learning is widely used to mine large, unlabeled datasets to make data-driven discoveries in critical domains such as climate science, biomedicine, astronomy, chemistry, and more. However, despite its widespread…
Data archiving is one of the most critical issues for modern astronomical observations. With the development of a new generation of radio telescopes, the transfer and archiving of massive remote data have become urgent problems to be…
Big data dictate their requirements to the hardware and software. Simple migration to the cloud data processing, while solving the problem of increasing computational capabilities, however creates some issues: the need to ensure the safety,…
Ever since the era of internet had ushered in cloud computing, there had been increase in the demand for the unlimited data available through cloud computing for data analysis, pattern recognition and technology advancement. With this also…
Retrieving and analyzing transit feeds relies on working with analytical workflows that can handle the massive volume of data streams that are relevant to understand the dynamics of transit networks which are entirely deterministic in the…
The Square Kilometre Array Observatory (SKAO) faces unprecedented technological challenges due to the vast scale and complexity of its data. This paper provides an overview of research by the AMIGA group to address these computing and…
We report here on a project that has developed a practical approach to processing all-sky image collections on cloud platforms, using as an exemplar application the creation of three-color Hierarchical Progressive Survey (HiPS) maps of the…
Stellar streams are potentially a very sensitive observational probe of galactic astrophysics, as well as the dark matter population in the Milky Way. On the other hand, performing a detailed, high-fidelity statistical analysis of these…
In cosmology, the analysis of observational evidence is very important to test theoretical models of the Universe. Artificial neural networks are powerful and versatile computational tools for data modelling and are recently being…
By utilizing large-scale graph analytic tools implemented in the modern Big Data platform, Apache Spark, we investigate the topological structure of gravitational clustering in five different universes produced by cosmological $N$-body…
We review some aspects of the current state of data-intensive astronomy, its methods, and some outstanding data analysis challenges. Astronomy is at the forefront of "big data" science, with exponentially growing data volumes and data…
This paper presents a systematic literature review focusing on the application of machine learning techniques for deriving observational constraints in cosmology. The goal is to evaluate and synthesize existing research to identify…
The HIFI data processing pipeline was developed to systematically process diagnostic, calibration and astronomical observations taken with the HIFI science instrumentas part of the Herschel mission. The HIFI pipeline processed data from all…
Public cloud computing environments, such as Amazon AWS, Microsoft Azure, and the Google Cloud Platform, have achieved remarkable improvements in computational performance in recent years, and are also expected to be able to perform…