Related papers: Putting Data Science Pipelines on the Edge
As the field of data science continues to grow, there will be an ever-increasing demand for tools that make machine learning accessible to non-experts. In this paper, we introduce the concept of tree-based pipeline optimization for…
Training large language models (LLMs) now requires resources that exceed a single datacenter, making cross-datacenter strategies increasingly crucial. We present CrossPipe, a framework designed to optimize model training across…
Stream computation is one of the approaches suitable for FPGA-based custom computing due to its high throughput capability brought by pipelining with regular memory access. To increase performance of iterative stream computation, we can…
Real-time computation of data streams over affordable virtualized infrastructure resources is an important form of data in motion processing architecture. However, processing such data streams while ensuring strict guarantees on quality of…
The growing scale of data requires efficient memory subsystems with large memory capacity and high memory performance. Disaggregated architecture has become a promising solution for today's cloud and edge computing for its scalability and…
To mitigate global climate change, distributed energy resources (DERs), such as distributed generators, flexible loads, and energy storage systems (ESSs), have witnessed rapid growth in power distribution systems. When properly managed,…
In recent years, Deep Learning (DL) has found great success in domains such as multimedia understanding. However, the complex nature of multimedia data makes it difficult to develop DL-based software. The state-of-the art tools, such as…
In many sciences the increasing amounts of data are reaching the limit of established data handling and processing. With four large research centers of the German Helmholtz association the Large Scale Data Management and Analysis (LSDMA)…
Large Language Models (LLMs) are increasingly deployed in both latency-sensitive online services and cost-sensitive offline workloads. Co-locating these workloads on shared serving instances can improve resource utilization, but directly…
Continuous Integration and Continuous Deployment (CI/CD) pipelines are central to modern software development. In large organizations, the high volume of builds and tests creates bottlenecks, especially under shared infrastructure. This…
Inter-datacenter networks connect dozens of geographically dispersed datacenters and carry traffic flows with highly variable sizes and different classes. Adaptive flow routing can improve efficiency and performance by assigning paths to…
Enterprise data lakes often suffer from substantial amounts of duplicate and redundant data, with data volumes ranging from terabytes to petabytes. This leads to both increased storage costs and unnecessarily high maintenance costs for…
The Internet of Things (IoT) is an emerging technology paradigm where millions of sensors and actuators help monitor and manage, physical, environmental and human systems in real-time. The inherent closedloop responsiveness and decision…
Regional planning processes and associated redevelopment projects can be complex due to the vast amount of diverse data involved. However, all of this data shares a common geographical reference, especially in the renaturation of former…
Energy efficiency is a corner stone of sustainability in data center and high-performance networking. However, at present there is a notable structural mismatch between network silicon development targets and network equipment utilization…
The rising of the Cyber-Physical System (CPS) and the Industry 4.0 paradigms demands the design and the implementation of Digital Twin Frameworks (DTFs) that may support the quick build of reliable Digital Twins (DTs) for experimental and…
The ever-increasing amount, variety as well as generation and processing speed of today's data pose a variety of new challenges for developing Data-Intensive Software Systems (DISS). As with developing other kinds of software systems,…
We describe an ecosystem for teaching data science (DS) to engineers which blends theory, methods, and applications, developed at the Faculty of Physical and Mathematical Sciences, Universidad de Chile, over the last three years. This…
This work presents P4TE, an in-band traffic monitoring, load-aware packet forwarding, and flow rate controlling mechanism for traffic engineering in fat-tree topology-based data center networks using PISA switches. It achieves sub-RTT…
Real-time data processing applications with low latency requirements have led to the increasing popularity of stream processing systems. While such systems offer convenient APIs that can be used to achieve data parallelism automatically,…