Related papers: DataOps-driven CI/CD for analytics repositories
Healthcare data are generated in many different formats, which makes it difficult to integrate and reuse across institutions and studies. Standardisation is required to enable consistent large-scale analysis. The OMOP-CDM, developed by the…
In the era of big data, conventional RDBMS models have become impractical for handling colossal workloads. Consequently, NoSQL databases have emerged as the preferred storage solutions for executing processing-intensive Online Analytical…
GitOps is a foundational approach for modernizing infrastructure by leveraging Git as the single source of truth for declarative configurations. The poster explores how GitOps transforms traditional control system infrastructure, services…
Synthetic data has emerged as a crucial solution to the data scarcity bottleneck in large language models (LLMs), particularly for specialized domains and low-resource languages. However, the broader adoption of existing synthetic data…
Massive, multi-language, monolithic repositories form the backbone of many modern, complex software systems. To ensure consistent code quality while still allowing fast development cycles, Continuous Integration (CI) is commonly applied.…
Porting a scientific data analysis workflow (DAW) to a cluster infrastructure, a new software stack, or even only a new dataset with some notably different properties is often challenging. Despite the structured definition of the steps…
Context: DevOps practices combine software development and IT operations. There is a growing number of DevOps related posts in popular online developer forum Stack Overflow (SO). While previous research analyzed SO posts related to…
DevOps pipeline is a set of automated tasks or processes or jobs that has tasks assigned to execute automatically that allow the Development team and Operations team to collaborate for building and deployment of the software or services.…
Parallel dataflow systems have become a standard technology for large-scale data analytics. Complex data analysis programs in areas such as machine learning and graph analytics often involve control flow, i.e., iterations and branching.…
Data-centric AI has shed light on the significance of data within the machine learning (ML) pipeline. Recognizing its significance, academia, industry, and government departments have suggested various NLP data research initiatives. While…
Real-world enterprise data intelligence workflows encompass data engineering that turns raw sources into analytical-ready tables and data analysis that convert those tables into decision-oriented insights. We introduce DAComp, a benchmark…
The demand for rapid software delivery in the Information Technology (IT) industry has significantly intensified, emphasising the need for faster software products and service releases with enhanced features to meet customer expectations.…
The nature and complexity of software have changed significantly in the last few decades. With the easy availability of computing power, deeper and broader applications are made. It has been extremely necessary to produce good quality…
Validation of conformance to cybersecurity standards for industrial automation and control systems is an expensive and time consuming process which can delay the time to market. It is therefore crucial to introduce conformance validation…
Digital business transformation has become increasingly important for organizations. Since transforming business digitally is an ongoing process, it requires an integrated and disciplined approach. Data Operations (DataOps), emerging in…
Software as a Service (SaaS) pricing models, encompassing features, usage limits, plans, and add-ons, have grown exponentially in complexity, evolving from offering tens to thousands of configuration options. This rapid expansion poses…
Continuous Integration and Continuous Deployment (CI/CD) pipeline automates software development to speed up and enhance the efficiency of engineering software. These workflows consist of various jobs, such as code validation and testing,…
Automated, data-driven quality management systems, which facilitate the transformation of data into useable information, are desired to enhance decision-making processes. Integration of accurate, reliable, and straightforward approaches…
Model deployment in machine learning has emerged as an intriguing field of research in recent years. It is comparable to the procedure defined for conventional software development. Continuous Integration and Continuous Delivery (CI/CD)…
The advancements in the software industry, along with the changing technologies, methods, and conditions, have particularly brought forth a perspective that prioritizes the improvement of all stages of the software development lifecycle by…