MPCDF HPC Performance Monitoring System: Enabling Insight via Job-Specific Analysis

Luka Stanisic; Klaus Reuter

MPCDF HPC Performance Monitoring System: Enabling Insight via Job-Specific Analysis

Distributed, Parallel, and Cluster Computing 2019-09-27 v1

Authors: Luka Stanisic , Klaus Reuter

Abstract

This paper reports on the design and implementation of the HPC performance monitoring system deployed to continuously monitor performance metrics of all jobs on the HPC systems at the Max Planck Computing and Data Facility (MPCDF). Thereby it reveals important information to various stakeholders, in particular to users, application support, system administrators, and management. On each compute node, hardware and software performance monitoring data is collected by our newly developed lightweight open-source hpcmd middleware which builds upon standard Linux tools. The data is transported via rsyslog, and aggregated and processed by a Splunk system, enabling detailed per-cluster and per-job interactive analysis in a web browser. Additionally, performance reports are provided to the users as PDF files. Finally, we report on practical experience and benefits from large-scale deployments on MPCDF HPC systems, demonstrating how our solution can be useful to any HPC center.

Keywords

hpc performance monitoring job-specific analysis hpcmd splunk rsyslog lightweight middleware max planck computing and data facility

Cite

@article{arxiv.1909.11704,
  title  = {MPCDF HPC Performance Monitoring System: Enabling Insight via Job-Specific Analysis},
  author = {Luka Stanisic and Klaus Reuter},
  journal= {arXiv preprint arXiv:1909.11704},
  year   = {2019}
}

Comments

13 pages, 3 figures

MPCDF HPC Performance Monitoring System: Enabling Insight via Job-Specific Analysis

Abstract

Keywords

Cite

Comments

Related papers