相关论文: Power efficient ReLU design for neuromorphic compu…

Current research suggests that the key factors in designing neural network architectures involve choosing number of filters for every convolution layer, number of hidden neurons for every fully connected layer, dropout and pruning. The…

This paper provides an analysis of state-of-the-art activation functions with respect to supervised classification of deep neural network. These activation functions comprise of Rectified Linear Units (ReLU), Exponential Linear Unit (ELU),…

Recent seminal work at the intersection of deep neural networks practice and random matrix theory has linked the convergence speed and robustness of these networks with the combination of random weight initialization and nonlinear…

Deep neural networks, particularly those employing Rectified Linear Units (ReLU), are often perceived as complex, high-dimensional, non-linear systems. This complexity poses a significant challenge to understanding their internal learning…

We propose a system for calculating a "scaling constant" for layers and weights of neural networks. We relate this scaling constant to two important quantities that relate to the optimizability of neural networks, and argue that a network…

Spin-Orbit Torque (SOT) Magnetic Random-Access Memory (MRAM) devices offer improved power efficiency, nonvolatility, and performance compared to static RAM, making them ideal, for instance, for cache memory applications. Efficient…

Current-induced spin-orbit torque (SOT) is regarded as a promising mechanism for driving neuromorphic behavior in spin-orbitronic devices. In principle, the strong SOT in heavy metal-based magnetic heterostructure is attributed to the…

Due to the limitations of realizing artificial neural networks on prevalent von Neumann architectures, recent studies have presented neuromorphic systems based on spiking neural networks (SNNs) to reduce power and computational cost.…

Spintronic artificial neurons are intriguing building blocks for energy efficient Neuromorphic Computing (NC). Nevertheless, most contemporary implementations rely on symmetry breaking external in plane magnetic fields (H_X) for neuron…

Transformers have improved drastically the performance of natural language processing (NLP) and computer vision applications. The computation of transformers involves matrix multiplications and non-linear activation functions such as…

Effectively manipulating magnetism in ferromagnet (FM) thin film nanostructures with an in-plane current has become feasible since the determination of a 'giant' spin Hall effect (SHE) in certain heavy metal (HM)/FM system. Recently, both…

Spin-memory loss (SML) of electrons traversing ferromagnetic-metal/heavy-metal (FM/HM), FM/normal-metal (FM/NM) and HM/NM interfaces is a fundamental phenomenon that must be invoked to explain consistently large number of spintronic…

LayerNorm is a critical component in modern large language models (LLMs) for stabilizing training and ensuring smooth optimization. However, it introduces significant challenges in mechanistic interpretability, outlier feature suppression,…

Nanomagnets driven by spin currents provide a natural implementation for a neuron and a synapse: currents allow convenient summation of multiple inputs, while the magnet provides the threshold function. The objective of this paper is to…

Spintronic technology is emerging as a direction for the hardware implementation of neurons and synapses of neuromorphic architectures. In particular, a single spintronic device can be used to implement the nonlinear activation function of…

We analyze the soft committee machine with Rectified Linear Unit (ReLU) activation by means of the replica method. In a realizable teacher--student setting, we compute the quenched free energy within a replica-symmetric ansatz and obtain…

Neuromorphic computing uses brain-inspired principles to design circuits that can perform computational tasks with superior power efficiency to conventional computers. Approaches that use traditional electronic devices to create artificial…

In artificial neural networks, neurons are usually implemented with highly dissipative CMOS-based operational amplifiers. A more energy-efficient implementation is a 'spin-neuron' realized with a magneto-tunneling junction (MTJ) that is…

Artificial modulation of a neuronal subset through ion channels activation can initiate firing patterns of an entire neural circuit in vivo. As nanovalves in the cell membrane, voltage-gated ion channels can be artificially controlled by…

In this paper, we develop a 6-input fracturable non-volatile Clockless LUT (C-LUT) using spin Hall effect (SHE)-based Magnetic Tunnel Junctions (MTJs) and provide a detailed comparison between the SHE-MTJ-based C-LUT and Spin Transfer…