Related papers: Optical activation function using a metamaterial w…
This paper explores the topological signatures of ReLU neural network activation patterns. We consider feedforward neural networks with ReLU activation functions and analyze the polytope decomposition of the feature space induced by the…
Dynamic adaptation in single-neuron response plays a fundamental role in neural coding in biological neural networks. Yet, most neural activation functions used in artificial networks are fixed and mostly considered as an inconsequential…
Activation function is a pivotal component of deep learning, facilitating the extraction of intricate data patterns. While classical activation functions like ReLU and its variants are extensively utilized, their static nature and…
Activation function is a key component in deep learning that performs non-linear mappings between the inputs and outputs. Rectified Linear Unit (ReLU) has been the most popular activation function across the deep learning community.…
Detecting traveling photons is an essential primitive for many quantum information processing tasks. We introduce a single-photon detector design operating in the microwave domain, based on a weakly nonlinear metamaterial where the…
Modern neural networks rely on generic activation functions (ReLU, GELU, SiLU) that ignore the mathematical structure inherent in scientific data. We propose Neuro-Symbolic Activation Discovery, a framework that uses Genetic Programming to…
We present a conceptually simple but effective funnel activation for image recognition tasks, called Funnel activation (FReLU), that extends ReLU and PReLU to a 2D activation by adding a negligible overhead of spatial condition. The forms…
This study introduces a novel activation function, characterized by a dynamic slope that adjusts throughout the training process, aimed at enhancing adaptability and performance in deep neural networks for computer vision tasks. The…
Effective activation functions introduce non-linear transformations, providing neural networks with stronger fitting capa-bilities, which help them better adapt to real data distributions. Huawei Noah's Lab believes that dynamic activation…
In the past few years, we have witnessed an increased interest in the use of 2D materials for the realization of hybrid photonic nonlinear waveguides. Although graphene has attracted most of the attention, other families of 2D materials…
Photonic neural networks have been considered as the promising candidates for next-generation neuromorphic computation, aiming to break both the power consumption wall and processing speed boundary of state-to-date digital computing…
Recently, coherent control of the optical response of thin films of matter in standing waves has attracted considerable attention, ranging from applications in excitation-selective spectroscopy and nonlinear optics to demonstrations of…
We present SiLU network constructions whose approximation efficiency depends critically on proper hyperparameter tuning. For the square function $x^2$, with optimally chosen shift $a$ and scale $\beta$, we achieve approximation error…
In this paper, we explore the concept of adding learn-able slope and mean shift parameters to an activation function to improve the total response region. The characteristics of an activation function depend highly on the value of…
We propose a thin, compact, broadband, polarization and angle insensitive metamaterial absorber based on a tungsten reflector, silicon spacer and a top pattern composed of a double square-like ring resonator utilizing nickel(Ni). In such a…
Strong amplification in integrated photonics is one of the most desired optical functionalities for computing, communications, sensing, and quantum information processing. Semiconductor gain and cubic nonlinearities, such as four-wave…
Optical microresonators are of paramount importance in photonic circuits requiring fine spectral filtering or resonant light recirculation. Key performance metrics improve with increasing resonance quality factor (Q) across all…
Well-known activation functions like ReLU or Leaky ReLU are non-differentiable at the origin. Over the years, many smooth approximations of ReLU have been proposed using various smoothing techniques. We propose new smooth approximations of…
In neural networks, non-linearity is introduced by activation functions. One commonly used activation function is Rectified Linear Unit (ReLU). ReLU has been a popular choice as an activation but has flaws. State-of-the-art functions like…
Advanced photonic nanostructures have enabled the maximization of synthetic chiroptic activities. The unique structuring of these building blocks has empowered chiral selective interactions with electromagnetic waves in plasmonic structures…