Related papers: Gradient Descent Algorithm Inspired Adaptive Time …
Wireless Sensor Networks (WSNs) contain numerous sensor nodes having limited power resource, which report sensed data to the Base Station (BS) that requires high energy usage. Many routing protocols have been proposed in this regard…
In this paper an improved version of the graded precision localization algorithm GRADELOC, called IGRADELOC is proposed. The performance of GRADELOC is dependent on the regions formed by the overlapping radio ranges of the nodes of the…
A wide range of optimization problems arising in machine learning can be solved by gradient descent algorithms, and a central question in this area is how to efficiently compress a large-scale dataset so as to reduce the computational…
Gradient-based iterative optimization methods are the workhorse of modern machine learning. They crucially rely on careful tuning of parameters like learning rate and momentum. However, one typically sets them using heuristic approaches…
We present a novel optimization-based decoding algorithm for LDPC codes that is suitable for hardware architectures specialized to feed-forward neural networks. The algorithm is based on the projected gradient descent algorithm with a…
Stochastic Gradient Descent is used for large datasets to train models to reduce the training time. On top of that data parallelism is widely used as a method to efficiently train neural networks using multiple worker nodes in parallel.…
Modern large scale machine learning applications require stochastic optimization algorithms to be implemented on distributed computational architectures. A key bottleneck is the communication overhead for exchanging information, such as…
Stochastic gradient descent (SGD) is a widely adopted iterative method for optimizing differentiable objective functions. In this paper, we propose and discuss a novel approach to scale up SGD in applications involving non-convex functions…
Communication overhead severely hinders the scalability of distributed machine learning systems. Recently, there has been a growing interest in using gradient compression to reduce the communication overhead of the distributed training.…
Due to their high computational complexity, deep neural networks are still limited to powerful processing units. To promote a reduced model complexity by dint of low-bit fixed-point quantization, we propose a gradient-based optimization…
In decentralized optimization over networks, synchronizing the updates of all nodes incurs significant communication overhead. For this reason, much of the recent literature has focused on the analysis and design of asynchronous…
The ever-increasing number of WSN deployments based on a large number of battery-powered, low-cost sensor nodes, which are limited in their computing and power resources, puts the focus of WSN time synchronization research on three major…
Stochastic Gradient Descent (SGD) is very useful in optimization problems with high-dimensional non-convex target functions, and hence constitutes an important component of several Machine Learning and Data Analytics methods. Recently there…
Sensor nodes are low cost, low power devices that are used to collect physical data and monitor environmental conditions from remote locations. Wireless Sensor Networks(WSN) are collection of sensor nodes, coordinating among themselves to…
Wireless Sensor Networks (WSNs) consist of large number of randomly deployed energy constrained sensor nodes. Sensor nodes have ability to sense and send sensed data to Base Station (BS). Sensing as well as transmitting data towards BS…
Decentralized distributed optimization over time-varying graphs (networks) is nowadays a very popular branch of research in optimization theory and consensus theory. One of the motivations to consider such networks is an application to…
Due to the limited generation and finite inertia, microgrid suffers from the large frequency and voltage deviation which can lead to system collapse. Thus, reliable load shedding to keep frequency stable is required. Wireless network,…
Many real-world data are sequentially collected over time and often exhibit skewed class distributions, resulting in imbalanced data streams. While existing approaches have explored several strategies, such as resampling and reweighting,…
Distributed synchronous stochastic gradient descent (S-SGD) has been widely used in training large-scale deep neural networks (DNNs), but it typically requires very high communication bandwidth between computational workers (e.g., GPUs) to…
To improve the off-sample generalization of classical procedures minimizing the empirical risk under potentially heavy-tailed data, new robust learning algorithms have been proposed in recent years, with generalized median-of-means…