English

Benchmarking Optimization Software with Performance Profiles

Mathematical Software 2007-05-23 v2
Authors: Elizabeth D. Dolan , Jorge J. Moré
View on arXiv ↗ PDF ↗

Abstract

We propose performance profiles-distribution functions for a performance metric-as a tool for benchmarking and comparing optimization software. We show that performance profiles combine the best features of other tools for performance evaluation.

Keywords

software quality

Cite

@article{arxiv.cs/0102001,
  title  = {Benchmarking Optimization Software with Performance Profiles},
  author = {Elizabeth D. Dolan and Jorge J. Moré},
  journal= {arXiv preprint arXiv:cs/0102001},
  year   = {2007}
}

Comments

13 pages plus title and toc pages

Related papers

View all related →
Optimization and Control · Mathematics
Nested Performance Profiles for Benchmarking Software
Rasoul Hekmati, Hanieh Mirhajianmoghadam
2020-01-31
Optimization and Control · Mathematics
Ten New Benchmarks for Optimization
Xin-She Yang
2023-09-06
Optimization and Control · Mathematics
A note on using performance and data profilesfor training algorithms
Margherita Porcelli, Philippe L. Toint
2021-05-31
Artificial Intelligence · Computer Science
A Literature Survey of Benchmark Functions For Global Optimization Problems
Momin Jamil, Xin-She Yang
2013-08-20
Machine Learning · Computer Science
Partial Rankings of Optimizers
Julian Rodemann, Hannah Blocher
2024-09-09
Optimization and Control · Mathematics
Best practices for comparing optimization algorithms
Vahid Beiranvand, Warren Hare, Yves Lucet
2017-09-26
Performance · Computer Science
Efficient Instrumentation for Performance Profiling
Edu Metz, Raimondas Lencevicius
2007-05-23
Neural and Evolutionary Computing · Computer Science
Towards Realistic Optimization Benchmarks: A Questionnaire on the Properties of Real-World Problems
Koen van der Blom, Timo M. Deist, Tea Tušar, Mariapia Marchi +4
2020-04-15
Software Engineering · Computer Science
Identifying Process Improvement Opportunities through Process Execution Benchmarking
Luka Abb, Majid Rafiei, Timotheus Kampik, Jana-Rebecca Rehse
2025-04-24
Programming Languages · Computer Science
How to benchmark: the Measure-Explain-Test-Improve loop
Gabriel Scherer
2026-05-05
Neural and Evolutionary Computing · Computer Science
Using Affine Combinations of BBOB Problems for Performance Assessment
Diederick Vermetten, Furong Ye, Carola Doerr
2023-03-09
Machine Learning · Statistics
Benchmarking Simulation-Based Inference
Jan-Matthis Lueckmann, Jan Boelts, David S. Greenberg, Pedro J. Gonçalves +1
2021-04-12
Software Engineering · Computer Science
Interpreting Performance Profiles with Deep Learning
Zhuoran Liu
2025-08-06
Databases · Computer Science
Object Database Benchmarks
Jerome Darmont
2017-01-27
Software Engineering · Computer Science
Tools for assessing and optimizing the energy requirements of high performance scientific computing software
Kai Diethelm
2016-11-03
Computational Engineering, Finance, and Science · Computer Science
Randomness as Reference: Benchmark Metric for Optimization in Engineering
Stefan Ivić, Siniša Družeta, Luka Grbčić
2026-04-17
Distributed, Parallel, and Cluster Computing · Computer Science
A Study on Performance Analysis Tools for Applications Running on Large Distributed Systems
Ajanta De Sarkar, Nandini Mukherjee
2010-06-15
Quantum Physics · Physics
Benchmarking the Operation of Quantum Heuristics and Ising Machines: Scoring Parameter Setting Strategies on Optimization Applications
David E. Bernal Neira, Robin Brown, Pratik Sathe, Filip Wudarski +3
2024-02-19
Software Engineering · Computer Science
Benchmarking as Empirical Standard in Software Engineering Research
Wilhelm Hasselbring
2021-05-04
Software Engineering · Computer Science
Continuous Performance Benchmarking Framework for ROOT
Oksana Shadura, Vassil Vassilev, Brian Paul Bockelman
2019-10-02
Performance · Computer Science
A Survey of Embedded Software Profiling Methodologies
Rajendra Patel, Arvind Rajwat
2013-12-12
Performance · Computer Science
A Continuous Benchmarking Infrastructure for High-Performance Computing Applications
Christoph Alt, Martin Lanser, Jonas Plewinski, Atin Janki +4
2024-06-12
Neural and Evolutionary Computing · Computer Science
MO-IOHinspector: Anytime Benchmarking of Multi-Objective Algorithms using IOHprofiler
Diederick Vermetten, Jeroen Rook, Oliver L. Preuß, Jacob de Nobel +4
2024-12-11
Distributed, Parallel, and Cluster Computing · Computer Science
MPI Implementation Profiling for Better Application Performance
Riley Shipley, Garrett Hooten, David Boehme, Derek Schafer +2
2024-02-20
Materials Science · Physics
Thermal Benchmark and Power Benchmark Software
M. Marcu, M. Vladutiu, H. Moldovan, M. Popa
2007-09-13