English

Instructional Level Parallelism

Hardware Architecture 2019-09-17 v1
Authors: Taposh Dutta-Roy
View on arXiv ↗ PDF ↗

Abstract

This paper is a review of the developments in Instruction level parallelism. It takes into account all the changes made in speeding up the execution. The various drawbacks and dependencies due to pipelining are discussed and various solutions to overcome them are also incorporated. It goes ahead in the last section to explain where is the new research leading us.

Keywords

parallel programming pipeline optimization machine learning

Cite

@article{arxiv.1909.06559,
  title  = {Instructional Level Parallelism},
  author = {Taposh Dutta-Roy},
  journal= {arXiv preprint arXiv:1909.06559},
  year   = {2019}
}

Comments

Branch Prediction, Exceptions, Instructional level parallelism, pipelining

Related papers

View all related →
Computation and Language · Computer Science
A Survey on Parallel Reasoning
Ziqi Wang, Boye Niu, Zipeng Gao, Zhi Zheng +9
2025-10-15
Machine Learning · Computer Science
Benchmarking Machine Learning: How Fast Can Your Algorithms Go?
Zeyu Ning, Hugues Nelson Iradukunda, Qingquan Zhang, Ting Zhu
2021-01-12
Machine Learning · Computer Science
A Survey and Empirical Evaluation of Parallel Deep Learning Frameworks
Daniel Nichols, Siddharth Singh, Shu-Huai Lin, Abhinav Bhatele
2022-07-04
Artificial Intelligence · Computer Science
Exploring Parallelism in Learning Belief Networks
TongSheng Chu, Yang Xiang
2013-02-08
Distributed, Parallel, and Cluster Computing · Computer Science
Model Parallelism on Distributed Infrastructure: A Literature Review from Theory to LLM Case-Studies
Felix Brakel, Uraz Odyurt, Ana-Lucia Varbanescu
2024-03-07
Machine Learning · Computer Science
Parallel Online Learning
Daniel Hsu, Nikos Karampatziakis, John Langford, Alex Smola
2011-03-23
Software Engineering · Computer Science
To Parallelize or Not to Parallelize, Speed Up Issue
Alaa Ismail Elnashar
2011-03-30
Distributed, Parallel, and Cluster Computing · Computer Science
Efficient Pipeline Planning for Expedited Distributed DNN Training
Ziyue Luo, Xiaodong Yi, Guoping Long, Shiqing Fan +3
2022-08-23
Computer Vision and Pattern Recognition · Computer Science
Faster Multi-GPU Training with PPLL: A Pipeline Parallelism Framework Leveraging Local Learning
Xiuyuan Guo, Chengqi Xu, Guinan Guo, Feiyu Zhu +5
2024-11-21
Computer Vision and Pattern Recognition · Computer Science
Massively Parallel Video Networks
Joao Carreira, Viorica Patraucean, Laurent Mazare, Andrew Zisserman +1
2018-09-06
Machine Learning · Computer Science
Exploring Hidden Dimensions in Parallelizing Convolutional Neural Networks
Zhihao Jia, Sina Lin, Charles R. Qi, Alex Aiken
2018-06-12
Machine Learning · Computer Science
Exploiting Parallelism Opportunities with Deep Learning Frameworks
Yu Emma Wang, Carole-Jean Wu, Xiaodong Wang, Kim Hazelwood +1
2020-07-01
High Energy Physics - Phenomenology · Physics
Comparison of methods for the calculation of the real dilogarithm regarding instruction-level parallelism
Alexander Voigt
2022-01-06
Machine Learning · Computer Science
The Impossibility of Parallelizing Boosting
Amin Karbasi, Kasper Green Larsen
2023-08-22
Artificial Intelligence · Computer Science
Parallelizing Probabilistic Inference: Some Early Explorations
Bruce D'Ambrosio, Tony Fountain, Zhaoyu Li
2013-03-25
Programming Languages · Computer Science
Cimple: Instruction and Memory Level Parallelism
Vladimir Kiriansky, Haoran Xu, Martin Rinard, Saman Amarasinghe
2018-07-05
Machine Learning · Computer Science
Parallel training of linear models without compromising convergence
Nikolas Ioannou, Celestine Dünner, Kornilios Kourtis, Thomas Parnell
2018-12-20
Distributed, Parallel, and Cluster Computing · Computer Science
Tools for Analyzing Parallel I/O
Julian M. Kunkel, Eugen Betke, Matt Bryson, Philip Carns +4
2018-07-19
Machine Learning · Computer Science
BitPipe: Bidirectional Interleaved Pipeline Parallelism for Accelerating Large Models Training
Houming Wu, Ling Chen, Wenjie Yu
2024-10-28
Distributed, Parallel, and Cluster Computing · Computer Science
Zero Bubble Pipeline Parallelism
Penghui Qi, Xinyi Wan, Guangxing Huang, Min Lin
2024-01-22
Distributed, Parallel, and Cluster Computing · Computer Science
Research on Model Parallelism and Data Parallelism Optimization Methods in Large Language Model-Based Recommendation Systems
Haowei Yang, Yu Tian, Zhongheng Yang, Zhao Wang +2
2025-06-25
Machine Learning · Computer Science
Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis
Tal Ben-Nun, Torsten Hoefler
2018-09-18
Machine Learning · Computer Science
Where Is My Training Bottleneck? Hidden Trade-Offs in Deep Learning Preprocessing Pipelines
Alexander Isenko, Ruben Mayer, Jeffrey Jedele, Hans-Arno Jacobsen
2022-03-28
Programming Languages · Computer Science
Extended abstract: Type oriented programming for task based parallelism
Nick Brown, Ludovic Capelli, J. Mark Bull
2020-10-28
Machine Learning · Computer Science
Embarrassingly Parallel Independent Training of Multi-Layer Perceptrons with Heterogeneous Architectures
Felipe Costa Farias, Teresa Bernarda Ludermir, Carmelo Jose Albanez Bastos-Filho
2022-06-20