相关论文: Power efficient ReLU design for neuromorphic compu…

ReLU neural-networks have been in the focus of many recent theoretical works, trying to explain their empirical success. Nonetheless, there is still a gap between current theoretical results and empirical observations, even in the case of…

Recent studies have shown that the choice of activation function can significantly affect the performance of deep learning networks. However, the benefits of novel activation functions have been inconsistent and task dependent, and…

Convolutional Neural Networks (CNNs) usually use the same activation function, such as RELU, for all convolutional layers. There are performance limitations of just using RELU. In order to achieve better classification performance, reduce…

Deep spiking neural networks (SNNs) offer the promise of low-power artificial intelligence. However, training deep SNNs from scratch or converting deep artificial neural networks to SNNs without loss of performance has been a challenge.…

We present artificial neural network design using spin devices that achieves ultra low voltage operation, low power consumption, high speed, and high integration density. We employ spin torque switched nano-magnets for modelling neuron and…

Due to the massive parallel computing capability and outstanding image and signal processing performance, cellular neural network (CNN) is one promising type of non-Boolean computing system that can outperform the traditional digital logic…

The choice of activation functions is crucial for modern deep neural networks. Popular hand-designed activation functions like Rectified Linear Unit(ReLU) and its variants show promising performance in various tasks and models. Swish, the…

As modern deep learning architectures grow in complexity, representational ambiguity emerges as a critical barrier to their interpretability and reliable merging. For ReLU networks, identical functional mappings can be achieved through…

Recent findings suggest that the consecutive layers of ReLU neural networks can be understood geometrically as space folding transformations of the input space, revealing patterns of self-similarity. In this paper, we present the first…

Fourier Analysis Network (FAN) was recently proposed as a simple way to improve neural network performance by replacing part of Rectified Linear Unit (ReLU) activations with sine and cosine functions. Although several studies have reported…

Rectified Linear Units (ReLU) are the default choice for activation functions in deep neural networks. While they demonstrate excellent empirical performance, ReLU activations can fall victim to the dead neuron problem. In these cases, the…

Spin-orbitronics, based on both spin and orbital angular momentum, presents a promising pathway for energy-efficient memory and logic devices. Recent studies have demonstrated the emergence of orbital currents in light transition metals…

Analog electronic non-volatile memories mimicking synaptic operations are being explored for the implementation of neuromorphic computing systems. Compound synapses consisting of ensembles of stochastic binary elements are alternatives to…

We report an intrinsic form of the inverse spin Hall effect (ISHE) in ferromagnetic (FM) metal with Rashba spin orbit coupling (RSOC), which is driven by a normal charge current. Unlike the conventional form, the ISHE can be induced without…

Spin Hall effect (SHE) induced reversal of perpendicular magnetization has attracted significant interest, due to its potential to lead to low power memory and logic devices. However, the switching requires an assisted in-plane magnetic…

Activation functions play a critical role in deep neural networks by shaping gradient flow, optimization stability, and generalization. While ReLU remains widely used due to its simplicity, it suffers from gradient sparsity and dead-neuron…

Neuromorphic engineering makes use of mixed-signal analog and digital circuits to directly emulate the computational principles of biological brains. Such electronic systems offer a high degree of adaptability, robustness, and energy…

Modern neural networks rely on generic activation functions (ReLU, GELU, SiLU) that ignore the mathematical structure inherent in scientific data. We propose Neuro-Symbolic Activation Discovery, a framework that uses Genetic Programming to…

Electrical control of the non-trivial topology in Weyl antiferromagnet is of great interests to develop next-generation spintronic devices. Recent works suggest that spin Hall effect can switch the topological antiferromagnetic order.…

The primary neural networks decision-making units are activation functions. Moreover, they evaluate the output of networks neural node; thus, they are essential for the performance of the whole network. Hence, it is critical to choose the…