Related papers: Revisiting Parameter Server in LLM Post-Training
Memory-to-memory data streaming is essential for modern scientific workflows that require near real-time data analysis, experimental steering, and informed decision-making during experiment execution. It eliminates the latency bottlenecks…
Although LLM training is typically centralized with high-bandwidth interconnects and large compute budgets, emerging methods target communication-constrained training in distributed environments. The model trade-offs introduced by this…
The growth of wireless communication technologies has been producing the intense demand for high-speed, efficient, reliable voice & data communication. As a result, third generation partnership project (3GPP) has implemented next generation…
In this paper, we study the optimal design for simultaneous wireless information and power transfer (SWIPT) in downlink multiuser orthogonal frequency division multiplexing (OFDM) systems. For information transmission, we consider two types…
The Future wireless communication systems face the challenging task of simultaneously providing high quality of service (QoS) and broadband data transmission, while also minimizing power consumption, latency, and system complexity. Although…
Bulk synchronous parallel (BSP) is the de-facto paradigm for distributed DNN training in today's production clusters. However, due to the global synchronization nature, its performance can be significantly influenced by network bottlenecks…
The sixth generation (6G) of wireless networks introduces integrated sensing and communication (ISAC), a technology in which communication and sensing functionalities are inextricably linked, sharing resources across time, frequency, space,…
Multiuser resource allocation has recently been recognized as an effective methodology for enhancing the power and spectrum efficiency in OFDM (orthogonal frequency division multiplexing) systems. It is, however, not directly applicable to…
The Data Distribution Service (DDS) is an Object Management Group (OMG) standard for high-performance and real-time systems. DDS is a data-centric middleware based on the publish-subscribe communication pattern and is used in many…
Large Language Models (LLMs) have become indispensable in numerous real-world applications. However, fine-tuning these models at scale, especially in federated settings where data privacy and communication efficiency are critical, presents…
Parameter updating is an important stage in parallelism-based distributed deep learning. Synchronous methods are widely used in distributed training the Deep Neural Networks (DNNs). To reduce the communication and synchronization overhead…
Over-the-air (OTA) computation has recently emerged as a communication-efficient Federated Learning (FL) paradigm to train machine learning models over wireless networks. However, its performance is limited by the device with the worst SNR,…
Model aggregation, the process that updates model parameters, is an important step for model convergence in distributed deep learning (DDL). However, the parameter server (PS), a popular paradigm of performing model aggregation, causes CPU…
Most existing work on adaptive allocation of subcarriers and power in multiuser orthogonal frequency division multiplexing (OFDM) systems has focused on homogeneous traffic consisting solely of either delay-constrained data (guaranteed…
Task-oriented communication is an emerging paradigm for next-generation communication networks, which extracts and transmits task-relevant information, instead of raw data, for downstream applications. Most existing deep learning (DL)-based…
The challenge of communication-efficient distributed optimization has attracted attention in recent years. In this paper, a communication efficient algorithm, called ordering-based alternating direction method of multipliers (OADMM) is…
This paper introduces OSC (Orchestrating Cognitive Synergy), a knowledge-aware adaptive collaboration framework designed to enhance cognitive synergy in multi-agent systems with large language models. While prior work has advanced agent…
Flooding protocols based on concurrent transmissions are regarded as the most reliable way to collect or disseminate data across a multi-hop low-power wireless mesh network. Recent works have shown that such protocols are effective for…
The demand for high-speed mobile wireless communications is rapidly growing. Orthogonal Frequency Division Multiplexing (OFDM) technology promises to be a key technique for achieving the high data capacity and spectral efficiency…
Resource allocation in Integrated Sensing and Communication (ISAC) systems is critical for balancing communication and sensing performance. This paper introduces a novel dynamic power allocation strategy for Orthogonal Frequency Division…