Related papers: Parallel Computation of PDFs on Big Spatial Data U…
This work explores the use of big data technologies deployed in the cloud for processing of astronomical data. We have applied Hadoop and Spark to the task of co-adding astronomical images. We compared the overhead and execution time of…
In this paper we demonstrate that multi-modal Probability Distribution Functions (PDFs) may be efficiently sampled using an algorithm originally developed for numerical integrations by Monte-Carlo methods. This algorithm can be used to…
The vast quantity of information brought by big data as well as the evolving computer hardware encourages success stories in the machine learning community. In the meanwhile, it poses challenges for the Gaussian process (GP) regression, a…
In probability density function (PDF) methods of turbulent flows, the joint PDF of several flow variables is computed by numerically integrating a system of stochastic differential equations for Lagrangian particles. A mathematically exact…
As the second component of SPARC (Simulation Package for Ab-initio Real-space Calculations), we present an accurate and efficient finite-difference formulation and parallel implementation of Density Functional Theory (DFT) for extended…
Different parallel frameworks for implementing data analysis applications have been proposed by the HPC and Big Data communities. In this paper, we investigate three task-parallel frameworks: Spark, Dask and RADICAL-Pilot with respect to…
In science and engineering, we often work with models designed for accurate prediction of variables of interest. Recognizing that these models are approximations of reality, it becomes desirable to apply multiple models to the same data and…
In modern spatial statistics, the structure of data that is collected has become more heterogeneous. Depending on the type of spatial data, different modeling strategies for spatial data are used. For example, a kriging approach for…
The joint probability density function (PDF) of turbulent velocity and concentration of a passive scalar in an urban street canyon is computed using a newly developed particle-in-cell Monte Carlo method. Compared to moment closures, the PDF…
Spatial data are central to applications such as environmental monitoring and urban planning, but are often distributed across devices where privacy and communication constraints limit direct sharing. Federated modeling offers a practical…
Despite the high accuracy of photometric redshifts (zphot) derived using Machine Learning (ML) methods, the quantification of errors through reliable and accurate Probability Density Functions (PDFs) is still an open problem. First, because…
At any given scale, 3$\times$2-point statistics extract only three numbers from the joint distribution of the cosmic matter density and galaxy density fluctuations: their variances and their covariance. It is well known that the full shape…
Querying very large RDF data sets in an efficient manner requires a sophisticated distribution strategy. Several innovative solutions have recently been proposed for optimizing data distribution with predefined query workloads. This paper…
Recently, due to rapid development of information and communication technologies, the data are created and consumed in the avalanche way. Distributed computing create preconditions for analyzing and processing such Big Data by distributing…
The probability density function (PDF) plays a central role in statistical and machine learning modeling. Real-world data often deviates from Gaussian assumptions, exhibiting skewness and exponential decay. To evaluate how well different…
In this paper, we study the statistical evolution of the large-scale structure (LSS), focusing on the joint probability distribution function (PDF) of the coarse-grained cosmic field and its role in constructing effective dynamics. As the…
A robust uncertainty estimate in global analyses of Parton Distribution Functions (PDFs) is essential at the Large Hadron Collider (LHC), especially in view of the high-precision data anticipated by experimentalists in the High-Luminosity…
Signal processing techniques will lean on blind methods in the near future, where no redundant, resource allocating information will be transmitted through the channel. To achieve a proper decision, however, it is essential to know at least…
With increasing point of interest (POI) datasets available with fine-grained spatial and temporal attributes, space-time Ripley's K function has been regarded as a powerful approach to analyze spatiotemporal point process. However,…
The need to analyze the available large synoptic multi-band surveys drives the development of new data-analysis methods. Photometric redshift estimation is one field of application where such new methods improved the results, substantially.…