相关论文: D0 Data Handling Operational Experience

Graphical Processing Units (GPUs) have recently become a valuable computing tool for the acquisition of data at high rates and for a relatively low cost. The devices work by parallelizing the code into thousands of threads, each executing a…

There has been a massive explosion of data generated by customers and retained by companies in the last decade. However, there is a significant mismatch between the increasing volume of data and the lack of automation methods and tools. The…

The four LHC experiments at CERN have decided to use a commercial SCADA (Supervisory Control And Data Acquisition) product for the supervision of their DCS (Detector Control System). The selected SCADA, which is therefore used for the CMS…

The time-of-flight (TOF) system in the Compressed Baryonic Matter (CBM) experiment is composed of super modules based on multi-gap Resistive Plate Chambers (MRPC) for high-denseness, high-resolution time measurement. In order to evaluate…

High-precision modeling of systems is one of the main areas of industrial data analysis. Models of systems, their digital twins, are used to predict their behavior under various conditions. We have developed several models of a storage…

Big data storage management is one of the most challenging issues for Grid computing environments, since large amount of data intensive applications frequently involve a high degree of data access locality. Grid applications typically deal…

AIS data from ships is excellent for analyzing single-ship movements and monitoring all ships within a specific area. However, the AIS data needs to be cleaned, processed, and stored before being usable. This paper presents a system…

Scientific applications produce a huge amount of data, which imposes serious management and analysis challenges. In particular, limitations in current database management systems prevent their adoption in simulation applications, in which…

Current distributed key value stores achieve scalability by trading off consistency. As persistent memory technologies evolve tremendously, it is not necessary to sacrifice consistency for performance. This paper proposes DTranx, a…

Distributed computing platforms provide a robust mechanism to perform large-scale computations by splitting the task and data among multiple locations, possibly located thousands of miles apart geographically. Although such distribution of…

This work presents novel distributed data collection systems and storage algorithms for collaborative learning wireless sensor networks (WSNs). In a large WSN, consider $n$ collaborative sensor devices distributed randomly to acquire…

Data is a precious resource in today's society, and is generated at an unprecedented and constantly growing pace. The need to store, analyze, and make data promptly available to a multitude of users introduces formidable challenges in…

Data-intensive applications often require exploratory analysis of large datasets. If analysis is performed on distributed resources, data locality can be crucial to high throughput and performance. We propose a "data diffusion" approach…

An extensible power supply control and monitoring system has been developed at Fermilab's Technical Division as a result of requirements to control and monior power supplies of various types from within many different applications. This…

In recent years, precision agriculture that uses modern information and communication technologies is becoming very popular. Raw and semi-processed agricultural data are usually collected through various sources, such as: Internet of Thing…

One of the purposes of Big Data systems is to support analysis of data gathered from heterogeneous data sources. Since data warehouses have been used for several decades to achieve the same goal, they could be leveraged also to provide…

Data processing frameworks such as Apache Beam and Apache Spark are used for a wide range of applications, from logs analysis to data preparation for DNN training. It is thus unsurprising that there has been a large amount of work on…

In-memory (transactional) data stores are recognized as a first-class data management technology for cloud platforms, thanks to their ability to match the elasticity requirements imposed by the pay-as-you-go cost model. On the other hand,…

This research developed a prototype data warehouse to integrate multi-source forestry data for long-term monitoring, management, and sustainability. The data warehouse is intended to accommodate all types of imagery from various platforms,…

The cost of using a blockchain infrastructure as well as the time required to search and retrieve information from it must be considered when designing a decentralized application. In this work, we examine a comprehensive set of data…