相关论文: Optical activation function using a metamaterial w…

This paper studies the role of activation functions in learning modular addition with two-layer neural networks. We first establish a sharp expressivity gap: sine MLPs admit width-$2$ exact realizations for any fixed length $m$ and, with…

Deep learning has been widely used in many fields, but the model training process usually consumes massive computational resources and time. Therefore, designing an efficient neural network training method with a provable convergence…

Resonant modes in metamaterials have been widely utilized to amplify the optical response of 2D materials for practical device applications. However, the high loss at the resonant mode severely hinders metamaterial applications. Here, we…

This paper introduces a significantly better class of activation functions than the almost universally used ReLU like and Sigmoidal class of activation functions. Two new activation functions referred to as the Cone and Parabolic-Cone that…

A metamaterial perfect absorber consisting of a tri-layer (Al/ZnS/Al) metal-dielectric-metal system with top aluminium nano-disks is fabricated by laser-interference lithography and lift-off processing. The metamaterial absorber had peak…

A general procedure for introducing parametric, learned, nonlinearity into activation functions is found to enhance the accuracy of representative neural networks without requiring significant additional computational resources. Examples…

The third-order optical nonlinearity in optical waveguides has found applications in optical switching, optical wavelength conversion, optical frequency comb generation, and ultrafast optical signal processing. The development of an…

We propose a novel activation function that implements piece-wise orthogonal non-linear mappings based on permutations. It is straightforward to implement, and very computationally efficient, also it has little memory requirements. We…

Graphene photonics has emerged as a promising platform for providing desirable optical functionality. However, graphene's monolayer-scale thickness fundamentally restricts the available light matter interaction, posing a critical design…

A well-known principle in optical physics states that power can never be exchanged between two light waves propagating inside a homogeneous medium when the medium response is strictly linear. Power exchange between light waves usually…

In recent years, deep neural networks (DNNs) achieved unprecedented performance in many low-level vision tasks. However, state-of-the-art results are typically achieved by very deep networks, which can reach tens of layers with tens of…

In the field of pattern recognition, achieving high accuracy is essential. While training a model to recognize different complex images, it is vital to fine-tune the model to achieve the highest accuracy possible. One strategy for…

Tremendous advances in image restoration tasks such as denoising and super-resolution have been achieved using neural networks. Such approaches generally employ very deep architectures, large number of parameters, large receptive fields and…

Brillouin scattering enables efficient and coherent conversion between optical photons and gigahertz-frequency phonons. Integrated circuits that harness this nonlinear interaction have immense potential for signal processing, quantum…

Recently integrated optics has become an intriguing platform for implementing machine learning algorithms and inparticular neural networks. Integrated photonic circuits can straightforwardly perform vector-matrix multiplicationswith high…

We present a simple, effective, and general activation function we term ACON which learns to activate the neurons or not. Interestingly, we find Swish, the recent popular NAS-searched activation, can be interpreted as a smooth approximation…

The nonlinearity of activation functions used in deep learning models are crucial for the success of predictive models. There are several commonly used simple nonlinear functions, including Rectified Linear Unit (ReLU) and Leaky-ReLU…

Activation function has a significant impact on the dynamics, convergence, and performance of deep neural networks. The search for a consistent and high-performing activation function has always been a pursuit during deep learning model…

Activation functions play a key role in providing remarkable performance in deep neural networks, and the rectified linear unit (ReLU) is one of the most widely used activation functions. Various new activation functions and improvements on…

Silicon photonics has attracted significant interest in recent years due to its potential in integrated photonics components (1,2) as well as all-dielectric meta-optics elements.(3) Strong photon-photon interactions, aka optical…