Related papers: Optical Transformers
Advances in deep neural networks (DNNs) are transforming science and technology. However, the increasing computational demands of the most powerful DNNs limit deployment on low-power devices, such as smartphones and sensors -- and this…
The recent push for post-Moore computer architectures has introduced a wide variety of application-specific accelerators. One particular accelerator, the resistance network analogue, has been well received due to its ability to efficiently…
Optics offers unique opportunities for reducing energy in information processing and communications while resolving the problem of interconnect bandwidth density inside machines. Such energy dissipation overall is now at environmentally…
In-SRAM computing promises energy efficiency, but circuit nonlinearities and PVT variations pose major challenges in designing robust accelerators. To address this, we introduce OPTIMA, a modeling framework that aids in analyzing bit-line…
Optical computing systems deliver unrivalled processing speeds for scalar operations. Yet, integrated implementations have been constrained to low-dimensional tensor operations that fall short of the vector dimensions required for modern…
Estimates of energy usage in layers of computing from devices to algorithms have been determined and analyzed. Building on the previous analysis [3], energy needed from single devices and systems including three large-scale computing…
Self-attention-based transformer models have achieved tremendous success in the domain of natural language processing. Despite their efficacy, accelerating the transformer is challenging due to its quadratic computational complexity and…
Optical neural networks (ONNs) have demonstrated record-breaking potential in high-performance neuromorphic computing due to their ultra-high execution speed and low energy consumption. However, current learning protocols fail to provide…
AI applications have emerged in current world. Among AI applications, computer-vision (CV) related applications have attracted high interest. Hardware implementation of CV processors necessitates a high performance but low-power image…
Optimization in machine learning, both theoretical and applied, is presently dominated by first-order gradient methods such as stochastic gradient descent. Second-order optimization methods, that involve second derivatives and/or second…
With its unique parallel processing capability, optical neural network has shown low-power consumption in image recognition and speech processing. At present, the manufacturing technology of programmable photonic chip is not mature, and the…
Large-scale artificial neural networks have shown significant promise in addressing a wide range of classification and recognition applications. However, their large computational requirements stretch the capabilities of computing…
Recent advances in machine learning (ML) have spotlighted the pressing need for computing architectures that bridge the gap between memory bandwidth and processing power. The advent of deep neural networks has pushed traditional Von Neumann…
Vision transformer based models bring significant improvements for image segmentation tasks. Although these architectures offer powerful capabilities irrespective of specific segmentation tasks, their use of computational resources can be…
This paper describes a memory-efficient transformer model designed to drive a reduction in memory usage and execution time by substantial orders of magnitude without impairing the model's performance near that of the original model.…
Convolutional neural networks are paramount in image and signal processing including the relevant classification and training tasks alike and constitute for the majority of machine learning compute demand today. With convolution operations…
Machine learning is becoming a widely used technique with a impressive growth due to the diversity of problem of societal interest where it can offer practical solutions. This increase of applications and required resources start to become…
We propose an optical memory cell based on ultrafast wavelength switching in coupled-cavity microlasers, featuring bistability between modes separated by several nanometers. A numerical implementation is demonstrated by simulating a…
The development of scalable quantum networks requires coherent interfaces capable of converting microwave photons used in superconducting quantum processors into optical photons suitable for long-distance fiber transmission. This review…
Tensor processing units (TPUs) are one of the most well-known machine learning (ML) accelerators utilized at large scale in data centers as well as in tiny ML applications. TPUs offer several improvements and advantages over conventional ML…