Related papers: Design and Execution of make-like, distributed Ana…
Integrated information theory provides a mathematical framework to fully characterize the cause-effect structure of a physical system. Here, we introduce PyPhi, a Python software package that implements this framework for causal analysis…
We introduce SurfFlow, an open-source high-throughput workflow package designed for automated first-principles calculations of surface energies in arbitrary crystals. Our package offers a comprehensive solution capable of handling…
Misconceptions about program execution hinder many novice programmers. We introduce SimpliPy, a notional machine designed around a carefully chosen Python subset to clarify core control flow and scoping concepts. Its foundation is a precise…
Scientific workflows automate the analysis of large-scale scientific data, fostering the reuse of data processing operators as well as the reproducibility and traceability of analysis results. In exploratory research, however, workflows are…
Common analysis techniques in multi-messenger astronomy involve hypothesis tests with unbinned log-likelihood (LLH) functions using data recorded in celestial coordinates to identify sources of high-energy cosmic particles in the Universe.…
The recently released Reflex scientific workflow environment supports the interactive execution of ESO VLT data reduction pipelines. Reflex is based upon the Kepler workflow engine, and provides components for organising the data, executing…
As real-world datasets become more complex and heterogeneous, supervised learning is often bottlenecked by input representation design. Modeling multimodal data, such as time-series, free text, and structured records, often requires…
Systematic exploration of hypotheses is a major part of any empirical research. In software engineering, we often produce unique tools for experiments and evaluate them independently on different data sets. In this paper, we present…
We argue that the Julia programming language is a compelling alternative to implementations in Python and C++ for common data analysis workflows in high energy physics. We compare the speed of implementations of different workflows in Julia…
High-level synthesis (HLS) enhances digital hardware design productivity through a high abstraction level. Even if the HLS abstraction prevents fine-grained manual register-transfer level (RTL) optimizations, it also enables automatable…
Empirical natural language processing (NLP) systems in application domains (e.g., healthcare, finance, education) involve interoperation among multiple components, ranging from data ingestion, human annotation, to text retrieval, analysis,…
With recent increasing computational and data requirements of scientific applications, the use of large clustered systems as well as distributed resources is inevitable. Although executing large applications in these environments brings…
Creating high-quality, large-scale datasets for large language models (LLMs) often relies on resource-intensive, GPU-accelerated models for quality filtering, making the process time-consuming and costly. This dependence on GPUs limits…
Statistical modeling is a key element in many scientific fields and especially in High-Energy Physics (HEP) analysis. The standard framework to perform this task in HEP is the C++ ROOT/RooFit toolkit; with Python bindings that are only…
Scientific workflow management systems offer features for composing complex computational pipelines from modular building blocks, for executing the resulting automated workflows, and for recording the provenance of data products resulting…
Researchers in the field of materials science, chemistry, and computational physics are regularly posed with the challenge of managing large and heterogeneous data spaces. The amount of data increases in lockstep with computational…
This work pursues automated planning and scheduling of distributed data pipelines, or workflows. We develop a general workflow and resource graph representation that includes both data processing and sharing components with corresponding…
Recent research in materials science opens exciting perspectives to design novel quantum materials and devices, but it calls for quantitative predictions of properties which are not accessible in standard first principles packages. PAOFLOW…
Scientific workflows consist of thousands of highly parallelized tasks executed in a distributed environment involving many components. Automatic tracing and investigation of the components' and tasks' performance metrics, traces, and…
Operationalizing AI has become a major endeavor in both research and industry. Automated, operationalized pipelines that manage the AI application lifecycle will form a significant part of tomorrow's infrastructure workloads. To optimize…