Related papers: Optical activation function using a metamaterial w…
Consider the multivariate nonparametric regression model. It is shown that estimators based on sparsely connected deep neural networks with ReLU activation function and properly chosen network architecture achieve the minimax rates of…
Nonlinear activation functions are widely recognized for enhancing the expressivity of neural networks, which is the primary reason for their widespread implementation. In this work, we focus on ReLU activation and reveal a novel and…
In this report we demonstrate that with same parameters and computational budgets, models with wider features before ReLU activation have significantly better performance for single image super-resolution (SISR). The resulted SR residual…
Recent studies have shown that the choice of activation function can significantly affect the performance of deep learning networks. However, the benefits of novel activation functions have been inconsistent and task dependent, and…
We present a guide for the design and fabrication of a CMOS-compatible metamaterial microstructure as an absorber of visible light with exceptionally high absorption efficiency (~ 98%), for wavelengths 400nm-700nm. The structural parameters…
Activation functions are essential to deep learning networks. Popular and versatile activation functions are mostly monotonic functions, some non-monotonic activation functions are being explored and show promising performance. But by…
Metamaterials offer unprecedented flexibility for manipulating the optical properties of matter, including the ability to access negative index, ultra-high index and chiral optical properties. Recently, metamaterials with near-zero…
We consider light propagation through a pair of nonlinear optical waveguides with absorption, placed in a medium with power gain. The active medium boosts the in-phase component of the overlapping evanescent fields of the guides, while the…
A phototransistor is a promising candidate as an optical power monitor in Si photonic circuits since the internal gain of photocurrent enables high sensitivity. However, state-of-the-art waveguide-coupled phototransistors suffer from a…
Gradient descent-based backpropagation training is widely used in many neural network systems. However, photonic implementation of such method is not straightforward mainly since having both the nonlinear activation function and its…
Structuring optical materials on a nanometer scale can lead to artificial effective media, or metamaterials, with strongly altered optical behavior. Metamaterials can provide a wide range of linear optical properties such as negative…
We propose deep-subwavelength optical waveguides based on metal-dielectric multilayer indefinite metamaterials with ultrahigh effective refractive indices. Waveguide modes with different mode orders are systematically analyzed with…
Optical parametric amplification (OPA) represents a powerful solution to achieve broadband amplification in wavelength ranges beyond the scope of conventional gain media, for generating high-power optical pulses, optical microcombs,…
Convolutional neural networks have been successful in solving many socially important and economically significant problems. This ability to learn complex high-dimensional functions hierarchically can be attributed to the use of nonlinear…
Stimulated Brillouin scattering in integrated photonic waveguides enables coherent coupling between optical photons and gigahertz acoustic phonons, providing a powerful mechanism for on-chip microwave photonics and opto-acoustic signal…
Metamaterials, artificial media structured on the subwavelength scale offer a rich paradigm for developing unique photonic functionalities ranging from negative index of refraction and directionally asymmetric transmission to slowing light.…
This paper proposes a novel nonlinear activation mechanism typically for convolutional neural network (CNN), named as reborn mechanism. In sharp contrast to ReLU which cuts off the negative phase value, the reborn mechanism enjoys the…
In this work, we present numerical results concerning an integrated photonic non-linear activation function that relies on a power independent, non-linear phase to amplitude conversion in a passive optical resonator. The underlying…
The Rectified Linear Unit (ReLU) is a foundational activation function in artficial neural networks. Recent literature frequently misattributes its origin to the 2018 (initial) version of this paper, which exclusively investigated ReLU at…
We investigate the saturable absorption behavior of a 1T'-MoTe$_2$ monolayer integrated with a silicon nitride waveguide for applications in photonic neural networks. Using experimental transmission measurements and theoretical modeling, we…