Related papers: Integrated Data Acquisition, Storage, Retrieval an…

BOSS (Batch Object Submission System) has been developed to provide real-time monitoring and bookkeeping of jobs submitted to a compute farm system. The information is persistently stored in a relational database (MySQL in the current…

Cloud computing has made federated database systems (FDBS) significantly more practical to implement than in the past. As part of a recent Web-based Geographic Information System (WebGIS) project, we are employing cloud-native technologies…

Getting the best performance from the ever-increasing number of hardware platforms has been a recurring challenge for data processing systems. In recent years, the advent of data science with its increasingly numerous and complex types of…

Relational database management systems (RDBMS) are widely used for the storage of structured data. To derive insights beyond statistical aggregation, we typically have to extract specific subdatasets from the database using conventional…

Here we present CaosDB, a Research Data Management System (RDMS) designed to ensure seamless integration of inhomogeneous data sources and repositories of legacy data. Its primary purpose is the management of data from biomedical sciences,…

In this article we describe the migration of event data collected by the COMPASS and HARP experiments at CERN. Together these experiments have over 300TB of physics data stored in Objectivity/DB that had to be transferred to a new data…

This paper introduces Web3DB, a decentralized relational database management system (RDBMS) designed to align with the principles of Web 3.0, addressing critical shortcomings of traditional centralized DBMS, such as data privacy, security…

A combined Doppler backscattering/cross-polarization scattering (DBS/CPS) system is being deployed on MAST-U, for simultaneous measurements of local density turbulence, turbulence flows, and magnetic turbulence. In this design, CPS shares…

High energy physics experiments in KEK/Japan rush into over KHz trigger stage. Thus, we need a successor of the data acquisition(DAQ) system that replaces the CAMAC or FASTBUS systems. To meet these needs, we have developed a DAQ system…

Existing data collection methods for traffic operations and control usually rely on infrastructure-based loop detectors or probe vehicle trajectories. Connected and automated vehicles (CAVs) not only can report data about themselves but…

Large-scale video repositories are increasingly available for modern video understanding and generation tasks. However, transforming raw videos into high-quality, task-specific datasets remains costly and inefficient. We present DataCube,…

Data Lake (DL) is a Big Data analysis solution which ingests raw data in their native format and allows users to process these data upon usage. Data ingestion is not a simple copy and paste of data, it is a complicated and important phase…

Cyber-physical systems (CPS) are being increasingly utilized for critical applications. CPS combines sensing and computing elements, often having multi-layer designs with networking, computational, and physical interfaces, which provide…

Autonomous driving has attracted tremendous attention especially in the past few years. The key techniques for a self-driving car include solving tasks like 3D map construction, self-localization, parsing the driving road and understanding…

As integrated sensing and communication (ISAC) becomes an integral part of 6G networks, distributed ISAC (DISAC) is expected to enhance both sensing and communication performance through its decentralized architecture. This paper presents a…

Embedded systems are playing an increasingly important role in control engineering. Despite their popularity, embedded systems are generally subject to resource constraints and it is therefore difficult to build complex control systems on…

The Network-on-chip (NoC) designs consisting of large pack of Intellectual Property (IP) blocks (cores) on the same silicon die is becoming technically possible nowadays. But, the communication between the IP Cores is the main issue in…

Data is the lifeblood of the modern world, forming a fundamental part of AI, decision-making, and research advances. With increase in interest in data, governments have taken important steps towards a regulated data world, drastically…

Caribou is a versatile data acquisition (DAQ) system developed within several collaborative frameworks (CERN EP R&D, DRD3, AIDAinnova, and Tangerine) to support laboratory and test-beam characterization of novel silicon pixel detectors. It…

Modern distributed databases face challenges in achieving transactional consistency across distributed partitions. Traditional two-phase commit (2PC) protocols incur high coordination overhead and latency, and require complex recovery for…