Related papers: DataOps-driven CI/CD for analytics repositories
In regulated domains such as finance, the integrity and governance of data pipelines are critical - yet existing systems treat data quality control (QC) as an isolated preprocessing step rather than a first-class system component. We…
Data quality assessment has become a prominent component in the successful execution of complex data-driven artificial intelligence (AI) software systems. In practice, real-world applications generate huge volumes of data at speeds. These…
In the last decade, companies adopted DevOps as a fast path to deliver software products according to customer expectations, with well aligned teams and in continuous cycles. As a basic practice, DevOps relies on pipelines that simulate…
Scientists are increasingly leveraging advances in instruments, automation, and collaborative tools to scale up their experiments and research goals, leading to new bursts of discovery. Various scientific disciplines, including…
With the increasing adoption of Continuous Integration and Continuous Deployment pipelines, securing software supply chains has become a critical challenge for modern DevOps teams. This study addresses these challenges by applying a…
DevOps is a combination of methodologies and tools that improves the software development, build, deployment, and monitoring processes by shortening its lifecycle and improving software quality. Part of this process is CI/CD, which embodies…
Operational rigor determines whether human-agent collaboration succeeds or fails. Scientific data pipelines need the equivalent of DevOps -- SciOps -- yet common approaches fragment provenance across disconnected systems without…
Critical goals of scientific computing are to increase scientific rigor, reproducibility, and transparency while keeping up with ever-increasing computational demands. This work presents an integrated framework well-suited for data…
Companies struggle to continuously develop and deploy AI models to complex production systems due to AI characteristics while assuring quality. To ease the development process, continuous pipelines for AI have become an active research area…
CI/CD pipelines are central to DevOps practices, yet their growing complexity makes them increasingly difficult to interpret, analyze, and systematically evolve. Existing tooling primarily offers execution logs and static graph…
This study evaluates the adoption of DevSecOps among small and medium-sized enterprises (SMEs), identifying key challenges, best practices, and future trends. Through a mixed methods approach backed by the Technology Acceptance Model (TAM)…
The rapidly growing demand for high-quality data in Large Language Models (LLMs) has intensified the need for scalable, reliable, and semantically rich data preparation pipelines. However, current practices remain dominated by ad-hoc…
Transparency is one of the most important principles of modern privacy regulations, such as the GDPR or CCPA. To be compliant with such regulatory frameworks, data controllers must provide data subjects with precise information about the…
Traditionally, promoted by the internet companies, continuous delivery is more and more appealing to industries which develop systems with safety-critical functions. Since safety-critical systems must meet regulatory requirements and…
Modern DevOps practices have accelerated software delivery through automation, CI/CD pipelines, and observability tooling,but these approaches struggle to keep pace with the scale and dynamism of cloud-native systems. As telemetry volume…
We outline a comprehensive framework for artificial intelligence (AI) Application Operations (AIAppOps), based on real-world experiences from diverse organizations. Data-driven projects pose additional challenges to organizations due to…
Software services play a crucial role in daily life, with automated actions determining access to resources and information. Trusting service providers to perform these actions fairly and accurately is essential, yet challenging for users…
Various data consistency levels have an important part in the integrity of data and also affect performance especially the data that is replicated many times across or over the cluster. Based on BASE and the theorem of CAP tradeoffs, most…
Real-world processes often involve interdependent objects that also carry data values, such as integers, reals, or strings. However, existing process formalisms fall short to combine key modeling features, such as tracking object…
Traditionally the integration of data from multiple sources is done on an ad-hoc basis for each analysis scenario and application. This is a solution that is inflexible, incurs in high costs, leads to "silos" that prevent sharing data…