分布式、并行与集群计算

Monolithic serving with chunked prefill improves GPU utilization by batching prefill and decode together, but suffers from fine-grained phase interference. Engine-level prefill-decode (PD) disaggregation avoids interference but incurs…

With the advancement of Artificial Intelligence (AI) towards multiple modalities (language, vision, speech, etc.), multi-modal models have increasingly been used across various applications (e.g., visual question answering or image…

Federated Learning (FL) enables collaborative model training on decentralized data but remains vulnerable to gradient leakage attacks that can reconstruct sensitive user information. Existing defense mechanisms, such as differential privacy…

In the past decade, blockchain has emerged as a promising solution for building secure distributed ledgers and has attracted significant attention. However, current blockchain systems suffer from limited throughput, poor scalability, and…

With the popularity of smart terminals, such as the Internet of Things, crowdsensing is an emerging data aggregation paradigm, which plays a pivotal role in data-driven applications. There are some key issues in the development of…

The increasing complexity of deep learning recommendation models (DLRM) has led to a growing need for large-scale distributed systems that can efficiently train vast amounts of data. In DLRM, the sparse embedding table is a crucial…

Nowadays, communication bottlenecks have emerged as a critical challenge in the distributed training and deployment of large language models (LLMs). This paper introduces FlashCommunication V2, a novel communication paradigm enabling…

A unit ball graph consists of a set of vertices, labeled by points in Euclidean space, and edges joining all pairs of points within distance 1. These geometric graphs are used to model a variety of spatial networks, including communication…

Scientific workflows facilitate the automation of data analysis, and are used to process increasing amounts of data. Therefore, they tend to be resource-intensive and long-running, leading to significant energy consumption and carbon…

Low-density parity-check (LDPC) codes are an important feature of several communication and storage applications, offering a flexible and effective method for error correction. These codes are computationally complex and require the…

In order to develop solutions that perform actions as early as possible, analysis of distributed algorithms using epistemic logic has generally concentrated on ``full information protocols'', which may be inefficient with respect to space…

In multi-task adversarial networks, the accurate estimation of unknown parameters in a distributed algorithm is hindered by attacked nodes or links. To tackle this challenge, this brief proposes a low-communication resilient distributed…

Fine-tuning large language models (LLMs) often exceeds GPU memory limits, prompting systems to offload model states to CPU memory. However, existing offloaded training frameworks like ZeRO-Offload treat all parameters equally and update the…

Simplicial complexes are a versatile and convenient paradigm on which to build all the tools and techniques of the logic of knowledge, on the assumption that initial epistemic models can be described in a distributed fashion. Thus, we can…

Blockchain is a decentralised, immutable ledger technology that has been widely adopted in many sectors for various applications such as cryptocurrencies, smart contracts and supply chain management. Distributed consensus is a fundamental…

Blockchain is a distributed ledger technology that has applications in many domains such as cryptocurrency, smart contracts, supply chain management, and many others. Distributed consensus is a fundamental component of blockchain systems…

A key challenge in on-chip interconnect design is to scale up bandwidth while maintaining low latency and high area efficiency. 2D-meshes scale with low wiring area and congestion overhead; however, their end-to-end latency increases with…

Hybrid transaction/analytical processing (HTAP) is an emerging database paradigm that supports both online transaction processing (OLTP) and online analytical processing (OLAP) workloads. Computing-intensive OLTP operations, involving…

With an ever growing number of heterogeneous applicational services running on equally heterogeneous computational systems, the problem of resource management becomes more essential. Although current solutions consider some network and time…

Federated Learning (FL) has emerged as a potential distributed learning paradigm that enables model training on edge devices (i.e., workers) while preserving data privacy. However, its reliance on a centralized server leads to limited…