Related papers: scida: scalable analysis for scientific big data
The usage of the high-level scripting language Python has enabled new mechanisms for data interrogation, discovery and visualization of scientific data. We present yt, an open source, community-developed astrophysical analysis and…
Karabo is a versatile Python-based software framework simplifying research with radio astronomy data. It bundles existing software packages into a coherent whole to improve the ease of use of its components. Karabo includes useful…
We make a case for "planetary computing" -- infrastructure to handle the ingestion, transformation, analysis and publication of global data products for furthering environmental science and enabling better informed policy-making. We draw on…
Wikidata is the largest collaborative general knowledge graph supported by a worldwide community. It includes many helpful topics for knowledge exploration and data science applications. However, due to the enormous size of Wikidata, it is…
Accurate simulation of astronomical observations is a critical element for any modern analyses, be it to measure event rates, analyses population properties, validate or train pipelines, account for selection effects, or correct biases. We…
The SunPy Project is a community of scientists and software developers creating an ecosystem of Python packages for solar physics. The project includes the sunpy core package as well as a set of affiliated packages. The sunpy core package…
Topological data analysis (TDA) is a powerful technique for extracting complex and valuable shape-related summaries of high-dimensional data. However, the computational demands of classical algorithms for computing TDA are exorbitant, and…
The Sand Atlas is a publicly accessible repository dedicated to the collection, processing, and sharing of high-resolution 3D models of sand-sized particles. This dataset offers valuable insights into the morphology of a wide variety of…
PSRCHIVE is an open-source, object-oriented, scientific data analysis software library and application suite for pulsar astronomy. It implements an extensive range of general-purpose algorithms for use in data calibration and integration,…
Intelligence analysts have long struggled with an abundance of data that must be investigated on a daily basis. In the U.S. Army, this activity involves reconciling information from various sources, a process that has been automated to a…
The sensitivity analysis and validation of simulation models require specific approaches in the case of spatial models. We describe the spatialdata scala library providing such tools, including synthetic generators for urban configurations…
SKA precursors are capable of detecting hundreds of galaxies in HI in a single 12 hours pointing. In deeper surveys one will probe more easily faint HI structures, typically located in the vicinity of galaxies, such as tails, filaments, and…
The recent influx of open scientific data has contributed to the transitioning of scientific computing from compute intensive to data intensive. Whereas many Big Data frameworks exist that minimize the cost of data transfers, few scientific…
Scientific writing involves retrieving, summarizing, and citing relevant papers, which can be time-consuming processes in large and rapidly evolving fields. By making these processes inter-operable, natural language processing (NLP)…
Graph transformations are a powerful computational model for manipulating complex networks, but handling temporal aspects and scalability remain significant challenges. We present a novel approach to implementing these transformations using…
We introduce STRAUSS (Sonification Tools and Resources for Analysis Using Sound Synthesis) a modular, self-contained and flexible Python sonification package, operating in a free and open source (FOSS) capacity. STRAUSS is intended to be a…
Spatial dataflow architectures like the Cerebras Wafer-Scale Engine deliver exceptional performance in AI and scientific computing by distributing scratchpad memory across hundreds of thousands of processing elements (PEs). Yet programming…
Reliable parameter extraction from experimental data is central to quantitative analysis in spectroscopy, diffraction, photoluminescence, chromatography, microscopy, and time-resolved measurements. We present FitED, a Python-based desktop…
Astrophysics has become a domain extremely rich of scientific data. Data mining tools are needed for information extraction from such large datasets. This asks for an approach to data management emphasizing the efficiency and simplicity of…
Computational materials science produces large quantities of data, both in terms of high-throughput calculations and individual studies. Extracting knowledge from this large and heterogeneous pool of data is challenging due to the wide…