分布式、并行与集群计算

This paper presents an empirical evaluation of the Proof of Team Sprint (PoTS) consensus algorithm, focusing on reward fairness, energy efficiency, system stability, and scalability. We conducted large-scale simulations comparing PoTS with…

In this work, we explore an object-based programming model for filling the space between shared memory and distributed systems programming. We argue that the natural representation for resources distributed across a memory network (e.g.…

In decentralized cloud computing marketplaces, ensuring fair and efficient interactions among asset providers and end-users is crucial. A key concern is meeting agreed-upon service-level objectives like the service's reliability. In this…

Various parallelism, such as data, tensor, and pipeline parallelism, along with memory optimizations like activation checkpointing, redundancy elimination, and offloading, have been proposed to accelerate distributed training for Large…

The increasing adoption of blockchain technology has led to a growing demand for higher transaction throughput. Traditional blockchain platforms, such as Ethereum, execute transactions sequentially within each block, limiting scalability.…

As Large Language Models (LLMs) continue to grow, reducing costs and alleviating GPU demands has become increasingly critical. However, existing schedulers primarily target either GPU compute or Key-Value Cache (KVC) utilization, failing to…

Serverless computing has emerged as a prominent paradigm, with a significant adoption rate among cloud customers. While this model offers advantages such as abstraction from the deployment and resource scheduling, it also poses limitations…

The deployment of Large Language Models (LLMs) on edge devices is increasingly important to enhance on-device intelligence. Weight quantization is crucial for reducing the memory footprint of LLMs on devices. However, low-bit LLMs…

Fine-tuning large language models (LLMs) greatly improves model quality for downstream tasks. However, serving many fine-tuned LLMs concurrently is challenging due to the sporadic, bursty, and varying request patterns of different LLMs. To…

Coordinating the design of sampling and sparse-dense matrix multiplication (SpMM) is crucial for accelerating graph neural networks (GNNs). However, due to irrational sampling strategies, existing methods face a trade-off between accuracy…

Energy is a fundamental component of modern life, driving nearly all aspects of daily activities. As such, the inability to access energy when needed is a significant issue that requires innovative solutions. In this paper, we propose…

Large language models (LLMs) are widely used but expensive to run, especially as inference workloads grow. To lower costs, maximizing the request batch size by managing GPU memory efficiently is crucial. While PagedAttention has recently…

Effective risk management solutions become absolutely crucial when financial markets embrace distributed technology and decentralized financing (DeFi). This study offers a thorough survey and comparative analysis of the integration of…

Photonics-based in-memory computing systems have demonstrated a significant speedup over traditional transistor-based systems because of their ultra-fast operating frequencies and high data bandwidths. Photonic static random access memory…

Sparse Matricized Tensor Times Khatri-Rao Product (spMTTKRP) is the bottleneck kernel of sparse tensor decomposition. In tensor decomposition, spMTTKRP is performed iteratively along all the modes of an input tensor. In this work, we…

Hardware memory disaggregation (HMD) is an emerging technology that enables access to remote memory, thereby creating expansive memory pools and reducing memory underutilization in datacenters. However, a significant challenge arises when…

Today's datacenter applications rely on datastores that are required to provide high availability, consistency, and performance. To achieve high availability, these datastores replicate data across several nodes. Such replication is managed…

In this work, we present WLB-LLM, a workLoad-balanced 4D parallelism for large language model training. We first thoroughly analyze the workload imbalance issue in LLM training and identify two primary sources of imbalance at the pipeline…

Virtual presence demands ultra-low latency, a factor that centralized architectures, by their nature, cannot minimize. Local peer-to-peer architectures offer a compelling alternative, but also pose unique challenges in terms of network…

Three-dimensional neutron transport calculations using the Method of Characteristics (MOC) are highly regarded for their exceptional computational efficiency, precision, and stability. Nevertheless, when dealing with extensive-scale…