Accelerating GenAI Workloads by Enabling RISC-V Microkernel Support in IREE

Adeel Ahmad; Ahmad Tameem Kamal; Nouman Amir; Bilal Zafar; Saad Bin Nasir

Accelerating GenAI Workloads by Enabling RISC-V Microkernel Support in IREE

Hardware Architecture 2025-08-22 v1 Artificial Intelligence

Authors: Adeel Ahmad , Ahmad Tameem Kamal , Nouman Amir , Bilal Zafar , Saad Bin Nasir

Abstract

This project enables RISC-V microkernel support in IREE, an MLIR-based machine learning compiler and runtime. The approach begins by enabling the lowering of MLIR linalg dialect contraction ops to linalg.mmt4d op for the RISC-V64 target within the IREE pass pipeline, followed by the development of optimized microkernels for RISC-V. The performance gains are compared with upstream IREE and Llama.cpp for the Llama-3.2-1B-Instruct model.

Keywords

computer architecture compiler optimization

Cite

@article{arxiv.2508.14899,
  title  = {Accelerating GenAI Workloads by Enabling RISC-V Microkernel Support in IREE},
  author = {Adeel Ahmad and Ahmad Tameem Kamal and Nouman Amir and Bilal Zafar and Saad Bin Nasir},
  journal= {arXiv preprint arXiv:2508.14899},
  year   = {2025}
}

Related papers

View all related →