Related papers: Entropy Mixing Networks: Enhancing Pseudo-Random N…
The effectiveness of deep neural networks (DNN) in vision, speech, and language processing has prompted a tremendous demand for energy-efficient high-performance DNN inference systems. Due to the increasing memory intensity of most DNN…
Modern scientific studies often collect data sets in the forms of tensors, which call for innovative statistical analysis methods. In particular, there is a pressing need for tensor clustering methods to understand the heterogeneity in the…
Entropy minimization (EM) is frequently used to increase the accuracy of classification models when they're faced with new data at test time. EM is a self-supervised learning method that optimizes classifiers to assign even higher…
The Expectation-Maximization (EM) algorithm for mixture models often results in slow or invalid convergence. The popular convergence proof affirms that the likelihood increases with Q; Q is increasing in the M -step and non-decreasing in…
We present a comprehensive software framework for the finite-size security analysis of quantum random number generation (QRNG) and quantum key distribution (QKD) protocols, based on the Entropy Accumulation Theorem (EAT). Our framework…
Information theoretic quantities play a central role in machine learning. The recent surge in the complexity of data and models has increased the demand for accurate estimation of these quantities. However, as the dimension grows the…
We propose an approximation of Echo State Networks (ESN) that can be efficiently implemented on digital hardware based on the mathematics of hyperdimensional computing. The reservoir of the proposed integer Echo State Network (intESN) is a…
The Matrix-based Renyi's entropy enables us to directly measure information quantities from given data without the costly probability density estimation of underlying distributions, thus has been widely adopted in numerous statistical…
Ensembling multiple predictions is a widely used technique for improving the accuracy of various machine learning tasks. One obvious drawback of ensembling is its higher execution cost during inference. In this paper, we first describe our…
Generative Adversarial Networks (GANs) can produce high-quality samples, but do not provide an estimate of the probability density around the samples. However, it has been noted that maximizing the log-likelihood within an energy-based…
Transformer-based entropy models have gained prominence in recent years due to their superior ability to capture long-range dependencies in probability distribution estimation compared to convolution-based methods. However, previous…
In this paper, we develop an information entropy based metric that represents the statistical quality of the generated binary sequence in Truly Random Number Generators (TRNG). The metric can be used for the design and optimization of the…
In quantum information geometry, the curvature of von-Neumann entropy and relative entropy induce a natural metric on the space of mixed quantum states. Here we use this information metric to construct a random matrix ensemble for states…
Despite the superiority of convolutional neural networks demonstrated in time series modeling and forecasting, it has not been fully explored on the design of the neural network architecture and the tuning of the hyper-parameters. Inspired…
Smart grids extremely rely on Information and Communications Technology (ICT) and smart meters to control and manage numerous parameters of the network. However, using these infrastructures make smart grids more vulnerable to cyber threats…
True random number generators (TRNGs) underpin modern cryptography, yet existing implementations face fundamental trade-offs between speed, scalability, and entropy quality. Here, we demonstrate that stochastic switching in the bistable…
Thompson sampling (TS) is a popular heuristic for action selection, but it requires sampling from a posterior distribution. Unfortunately, this can become computationally intractable in complex environments, such as those modeled using…
While deep neural networks are highly effective at solving complex tasks, large pre-trained models are commonly employed even to solve consistently simpler downstream tasks, which do not necessarily require a large model's complexity.…
Embedded devices are omnipresent in modern networks including the ones operating inside critical environments. However, due to their constrained nature, novel mechanisms are required to provide external, and non-intrusive anomaly detection.…
Cross-entropy method model predictive control (CEM--MPC) is a powerful gradient-free technique for nonlinear optimal control, but its performance is often limited by the reliance on random sampling. This conventional approach can lead to…