Related papers: Agile data management in NAV: A Case Study
Background: Distributed data-intensive systems are increasingly designed to be only eventually consistent. Persistent data is no longer processed with serialized and transactional access, exposing applications to a range of potential…
This paper addresses the problem of efficiently storing and accessing massive data blocks in a large-scale distributed environment, while providing efficient fine-grain access to data subsets. This issue is crucial in the context of…
Metadata management for distributed data sources is a long-standing but ever-growing problem. To counter this challenge in a research-data and library-oriented setting, this work constructs a data architecture, derived from the data-lake:…
The systems that operate the infrastructure of cities have evolved in a fragmented fashion across several generations of technology, causing city utilities and services to operate sub-optimally and limiting the creation of new value-added…
Nowadays, machine learning (ML) teams have multiple concurrent ML workflows for different applications. Each workflow typically involves many experiments, iterations, and collaborative activities and commonly takes months and sometimes…
Storage and memory systems for modern data analytics are heavily layered, managing shared persistent data, cached data, and non-shared execution data in separate systems such as distributed file system like HDFS, in-memory file system like…
The rise of multi-agent systems powered by large language models (LLMs) and specialized reasoning agents exposes fundamental limitations in today's data management architectures. Traditional databases and data fabrics were designed for…
The Internet of Things (IoT) data and social media data are two of the fastest-growing data segments. Having high-quality data is crucial for making informed business decisions. The strategic process of leveraging insights from data is…
The data warehouse (DW) technology was developed to integrate heterogeneous information sources for analysis purposes. Information sources are more and more autonomous and they often change their content due to perpetual transactions (data…
The exponential increase of availability of digital data and the necessity to process it in business and scientific fields has literally forced upon us the need to analyze and mine useful knowledge from it. Traditionally data mining has…
A growing trend in modern data analysis is the integration of data management with learning, guided by accuracy, latency, and cost requirements. In practice, applications draw data of different formats from many sources. In the meanwhile,…
Distributed software development is more difficult than co-located software development. One of the main reasons is that communication is more difficult in distributed settings. Defined processes and artifacts help, but cannot cover all…
Modern applications demand high performance and cost efficient database management systems (DBMSs). Their workloads may be diverse, ranging from online transaction processing to analytics and decision support. The cloud infrastructure…
Many large enterprises that operate highly governed and complex ICT environments have no efficient and effective way to support their Data and AI teams in rapidly spinning up and tearing down self-service data and compute infrastructure, to…
Nowadays agile software development is used in greater extend but for small organizations only, whereas MDA is suitable for large organizations but yet not standardized. In this paper the pros and cons of Model Driven Architecture (MDA) and…
As information becomes increasingly sizable for organizations to maintain the challenge of organizing data still remains. More importantly, the on-going process of analysing incoming data occurs on a continual basis and organizations should…
Context: User-Centered Design and Agile methodologies focus on human issues. Nevertheless, agile methodologies focus on contact with contracting customers and generating value for them. Usually, the communication between end users and the…
Centralized and distributed systems are two main approaches to organizing ICT infrastructure, each with its pros and cons. Centralized systems concentrate resources in one location, making management easier but creating single points of…
The web of data has brought forth the need to preserve and sustain evolving information within linked datasets; however, a basic requirement of data preservation is the maintenance of the datasets' structural characteristics as well. As…
A benchmark study of modern distributed databases is an important source of information to select the right technology for managing data in the cloud-edge paradigms. To make the right decision, it is required to conduct an extensive…