Related papers: Photonic Rails in ML Datacenters with Opus
Rail-optimized network fabrics have become the de facto datacenter scale-out fabric for large-scale ML training. However, the use of high-radix electrical switches to provide all-to-all connectivity in rails imposes massive power, cost, and…
The advancement of artificial intelligence demands flexible multimodal data processing with high throughput and energy efficiency. Photonic integrated circuits (PIC) has demonstrated promising potentials in terms of low latency and low…
This paper presents a low-cost network architecture for training large language models (LLMs) at hyperscale. We study the optimal parallelization strategy of LLMs and propose a novel datacenter network design tailored to LLM's unique…
Machine learning training places immense demands on cluster networks, motivating specialized architectures and co-design with parallelization strategies. Recent designs incorporating optical circuit switches (OCSes) are promising, offering…
With its unique parallel processing capability, optical neural network has shown low-power consumption in image recognition and speech processing. At present, the manufacturing technology of programmable photonic chip is not mature, and the…
Neuromorphic Computing implemented in photonic hardware is one of the most promising routes towards achieving machine learning processing at the picosecond scale, with minimum power consumption. In this work, we present a new concept for…
Data center networks are experiencing unprecedented exponential growth, mostly driven by the continuous computing demands in machine learning and artificial intelligence algorithms. Within this realm, optical networking offers numerous…
As computing resource demands continue to escalate in the face of big data, cloud-connectivity and the internet of things, it has become imperative to develop new low-power, scalable architectures. Neuromorphic photonics, or photonic neural…
Networks inside current data centers comprise a hierarchy of power-hungry electronic packet switches interconnected via optical fibers and transceivers. As the scaling of such electrically-switched networks approaches a plateau, a…
The rapid growth of AI training has dramatically increased datacenter traffic demand and energy consumption, which has motivated renewed interest in optical circuit switches (OCSes) as a high-bandwidth, energy-efficient alternative for AI…
Increasingly large AI workloads are calling for hyper-scale infrastructure; however, traditional interconnection network architecture is neither scalable nor cost-effective enough. Tree-based topologies such as the \textit{Rail-optimized}…
Distributed learning is widely used for training large models on large datasets by distributing parts of the model or dataset across multiple devices and aggregating the computed results for subsequent computations or parameter updates.…
As distributed machine learning (ML) workloads scale to thousands of GPUs connected by high-speed interconnects, tail latency in collective communication has become a major bottleneck. Existing RDMA transports, such as RoCE, IRN, SRNIC, and…
The need for optical parallelization is driven by the imminent optical capacity crunch, where the spectral efficiency required in the coming decades will be beyond the Shannon limit. To this end, the emerging high-speed Ethernet services at…
Communication through optical fibres experiences limitations due to chromatic dispersion and nonlinear Kerr effects that degrade the signal. Mitigating these impairments is typically done using complex digital signal processing algorithms.…
Neural networks are one of the disruptive computing concepts of our time. However, they fundamentally differ from classical, algorithmic computing in a number of fundamental aspects. These differences result in equally fundamental, severe…
High-speed signal processing is essential for maximizing data throughput in emerging communication applications, like multiple-input multiple-output (MIMO) systems and radio-frequency (RF) interference cancellation. However, as these…
Optimal Power Flow (OPF) refers to a wide range of related optimization problems with the goal of operating power systems efficiently and securely. In the simplest setting, OPF determines how much power to generate in order to minimize…
The Optimal Power Flow (OPF) problem is integral to the functioning of power systems, aiming to optimize generation dispatch while adhering to technical and operational constraints. These constraints are far from straightforward; they…
Fully Connected Neural Network (FCNN) is a class of Artificial Neural Networks widely used in computer science and engineering, whereas the training process can take a long time with large datasets in existing many-core systems. Optical…