分布式、并行与集群计算
Converged HPC-Cloud computing is an emerging computing paradigm that aims to support increasingly complex and multi-tenant scientific workflows. These systems require reconciliation of the isolation requirements of native cloud workloads…
The sparsely activated mixture-of-experts (MoE) transformer has become a common architecture for large language models (LLMs) due to its sparsity, which requires fewer computational demands while easily scaling the model size. In MoE…
Distributed machine learning training and inference is common today because today's large models require more memory and compute than can be provided by a single GPU. Distributed models are generally produced by programmers who take a…
Sharding is a technique to speed up transaction processing in blockchains, where the $n$ processing nodes in the blockchain are divided into $s$ disjoint groups (shards) that can process transactions in parallel. We study dynamic scheduling…
This paper presents Block, a distributed scheduling framework designed to optimize load balancing and auto-provisioning across instances in large language model serving frameworks by leveraging contextual information from incoming requests.…
Transformer models rely on High-Performance Computing (HPC) resources for inference, where soft errors are inevitable in large-scale systems, making the reliability of the model particularly critical. Existing fault tolerance frameworks for…
BLAS is a fundamental building block of advanced linear algebra libraries and many modern scientific computing applications. GPUs are known for their strong arithmetic computing capabilities and are highly suited for BLAS operations.…
We obtain improved distributed algorithms in the CONGEST message-passing setting for problems on power graphs of an input graph $G$. This includes Coloring, Maximal Independent Set, and related problems. We develop a general deterministic…
Serving disaggregated large language models has been widely adopted in industrial practice for enhanced performance. However, too many tokens generated in decoding phase, i.e., occupying the resources for a long time, essentially hamper the…
This paper addresses key challenges in task scheduling for multi-tenant distributed systems, including dynamic resource variation, heterogeneous tenant demands, and fairness assurance. An adaptive scheduling method based on reinforcement…
Accurate and adaptive network throughput prediction is essential for latency-sensitive and bandwidth-intensive applications in 5G and emerging 6G networks. However, most existing methods rely on centralized training with uniformly collected…
The proliferation of IoT devices and advancements in network technologies have intensified the demand for real-time data processing at the network edge. To address these demands, low-power AI accelerators, particularly GPUs, are…
Topological descriptors such as contour trees are widely utilized in scientific data analysis and visualization, with applications from materials science to climate simulations. It is desirable to preserve topological descriptors when data…
AI inference workflows are typically structured as a pipeline or graph of AI programs triggered by events. As events occur, the AIs perform inference or classification tasks under time pressure to respond or take some action. Standard…
Messaging systems are essential for efficiently transferring large volumes of data, ensuring rapid response times and high-throughput communication. The state-of-the-art on messaging systems mainly focuses on the performance evaluation of…
Modern hardware environments are becoming increasingly heterogeneous, leading to the emergence of applications specifically designed to exploit this heterogeneity. Efficiently adopting locks in these applications poses distinct challenges.…
6G network architectures will usher in a wave of innovative services and capabilities, introducing concepts like split computing and dynamic processing nodes. This implicates a paradigm where accessing resources seamlessly aligns with…
In this paper, we investigate: ``How can a group of initially co-located mobile agents perpetually explore an unknown graph, when one stationary node occasionally behaves maliciously, under an adversary's control?'' We call this node a…
In modern computer architectures, the performance of many memory-bound workloads (e.g., machine learning, graph processing, databases) is limited by the data movement bottleneck that emerges when transferring large amounts of data between…
Recent years have seen an unprecedented growth in research that leverages the newest computing paradigm of Internet of Drones, comprising a fleet of connected Unmanned Aerial Vehicles (UAVs) used for a wide range of tasks such as monitoring…