Related papers: Hardware Realization of Nonlinear Activation Funct…

This paper highlights new opportunities for designing large-scale machine learning systems as a consequence of blurring traditional boundaries that have allowed algorithm designers and application-level practitioners to stay -- for the most…

In this paper we present equivalence results for several types of unbounded operator functions. A generalization of the concept equivalence after extension is introduced and used to prove equivalence and linearization for classes of…

Quantization can improve the execution latency and energy efficiency of neural networks on both commodity GPUs and specialized accelerators. The majority of existing literature focuses on training quantized DNNs, while this work examines…

Convolutional Neural Networks (CNNs) have been widely applied. But as the CNNs grow, the number of arithmetic operations and memory footprint also increase. Furthermore, typical non-linear activation functions do not allow associativity of…

Proximal operators are fundamental across many applications in signal processing and machine learning, including solving ill-posed inverse problems. Recent work has introduced Learned Proximal Networks (LPNs), providing parametric functions…

Bayesian Neural Networks (BNNs) can overcome the problem of overconfidence that plagues traditional frequentist deep neural networks, and are hence considered to be a key enabler for reliable AI systems. However, conventional hardware…

Optical neural networks are at the forefront of computational innovation, utilizing photons as the primary carriers of information and employing optical components for computation. However, the fundamental nonlinear optical device in the…

All-optical diffractive neural networks (DNNs) offer a promising alternative to electronics-based neural network processing due to their low latency, high throughput, and inherent spatial parallelism. However, the lack of reconfigurability…

Future services demand high data rate and quality. Thus, it is necessary to define new and robust algorithms to equalize channels and reduce noise in communications. Nowadays, new equalization algorithms are being developed to optimize the…

In Transformer models, non-GEMM (non-General Matrix Multiplication) operations -- especially Softmax and Layer Normalization (LayerNorm) -- often dominate hardware cost due to their nonlinear nature. To address this, previous approximation…

This paper presents a simulation platform, namely CIMulator, for quantifying the efficacy of various synaptic devices in neuromorphic accelerators for different neural network architectures. Nonvolatile memory devices, such as resistive…

We propose an expectation-maximization-like(EMlike) method to train Boltzmann machine with unconstrained connectivity. It adopts Monte Carlo approximation in the E-step, and replaces the intractable likelihood objective with efficiently…

Sum-of-squares objective functions are very popular in computer vision algorithms. However, these objective functions are not always easy to optimize. The underlying assumptions made by solvers are often not satisfied and many problems are…

Functional near-infrared spectroscopy (fNIRS) is a non-invasive, low-cost method used to study the brain's blood flow pattern. Such patterns can enable us to classify performed by a subject. In recent research, most classification systems…

Structured convex optimization problems typically involve a mix of smooth and nonsmooth functions. The common practice is to activate the smooth functions via their gradient and the nonsmooth ones via their proximity operator. We show that,…

Binaural reproduction methods aim to recreate an acoustic scene for a listener over headphones, offering immersive experiences in applications such as Virtual Reality (VR) and teleconferencing. Among the existing approaches, the Binaural…

We report the possibility of using a simple neural network for effortless restoration of low-light images inspired by the retina model, which mimics the neurophysiological principles and dynamics of various types of optical neurons. The…

A crucial property for achieving secure, trustworthy and interpretable deep learning systems is their robustness: small changes to a system's inputs should not result in large changes to its outputs. Mathematically, this means one strives…

In this paper, we study approximation properties of single hidden layer neural networks with weights varying on finitely many directions and thresholds from an open interval. We obtain a necessary and at the same time sufficient measure…

Recently integrated optics has become an intriguing platform for implementing machine learning algorithms and inparticular neural networks. Integrated photonic circuits can straightforwardly perform vector-matrix multiplicationswith high…