English

COCO: The Experimental Procedure

Artificial Intelligence 2016-05-20 v2 Neural and Evolutionary Computing
Authors: Nikolaus Hansen , Tea Tusar , Olaf Mersmann , Anne Auger , Dimo Brockhoff
View on arXiv ↗ PDF ↗

Abstract

We present a budget-free experimental setup and procedure for benchmarking numericaloptimization algorithms in a black-box scenario. This procedure can be applied with the COCO benchmarking platform. We describe initialization of and input to the algorithm and touch upon therelevance of termination and restarts.

Cite

@article{arxiv.1603.08776,
  title  = {COCO: The Experimental Procedure},
  author = {Nikolaus Hansen and Tea Tusar and Olaf Mersmann and Anne Auger and Dimo Brockhoff},
  journal= {arXiv preprint arXiv:1603.08776},
  year   = {2016}
}

Comments

ArXiv e-prints, arXiv:1603.08776

Related papers

View all related →
Artificial Intelligence · Computer Science
COCO: A Platform for Comparing Continuous Optimizers in a Black-Box Setting
Nikolaus Hansen, Anne Auger, Raymond Ros, Olaf Mersmann +2
2020-09-10
Neural and Evolutionary Computing · Computer Science
COCO: Performance Assessment
Nikolaus Hansen, Anne Auger, Dimo Brockhoff, Dejan Tušar +1
2016-05-12
Neural and Evolutionary Computing · Computer Science
How Sequential Algorithm Portfolios can benefit Black Box Optimization
Catalin-Viorel Dinu, Diederick Vermetten, Carola Doerr
2026-01-26
Quantum Physics · Physics
Benchmarking the Operation of Quantum Heuristics and Ising Machines: Scoring Parameter Setting Strategies on Optimization Applications
David E. Bernal Neira, Robin Brown, Pratik Sathe, Filip Wudarski +3
2024-02-19
Quantum Physics · Physics
Towards Robust Benchmarking of Quantum Optimization Algorithms
David Bucher, Nico Kraus, Jonas Blenninger, Michael Lachner +2
2024-05-14
Neural and Evolutionary Computing · Computer Science
Benchmarking for Metaheuristic Black-Box Optimization: Perspectives and Open Challenges
Ramses Sala, Ralf Müller
2020-07-02
Mathematical Software · Computer Science
Blackbox: A procedure for parallel optimization of expensive black-box functions
Paul Knysh, Yannis Korkolis
2016-05-04
Optimization and Control · Mathematics
Best practices for comparing optimization algorithms
Vahid Beiranvand, Warren Hare, Yves Lucet
2017-09-26
Software Engineering · Computer Science
Employing Continuous Integration inspired workflows for benchmarking of scientific software -- a use case on numerical cut cell quadrature
Teoman Toprak, Michael Loibl, Guilherme H. Teixeira, Irina Shiskina +4
2026-05-01
Machine Learning · Computer Science
BBOPlace-Bench: Benchmarking Black-Box Optimization for Chip Placement
Ke Xue, Ruo-Tong Chen, Rong-Xi Tan, Xi Lin +4
2025-10-28
Machine Learning · Statistics
Automated Algorithm Selection on Continuous Black-Box Problems By Combining Exploratory Landscape Analysis and Machine Learning
Pascal Kerschke, Heike Trautmann
2018-11-30
Neural and Evolutionary Computing · Computer Science
MECHBench: A Set of Black-Box Optimization Benchmarks originated from Structural Mechanics
Iván Olarte Rodríguez, Maria Laura Santoni, Fabian Duddeck, Carola Doerr +2
2025-11-17
Machine Learning · Computer Science
Learning How to Optimize Black-Box Functions With Extreme Limits on the Number of Function Evaluations
Carlos Ansotegui, Meinolf Sellmann, Tapan Shah, Kevin Tierney
2021-03-19
Machine Learning · Computer Science
OpenBox: A Generalized Black-box Optimization Service
Yang Li, Yu Shen, Wentao Zhang, Yuanwei Chen +8
2021-11-05
Neural and Evolutionary Computing · Computer Science
Benchmarking Feature-based Algorithm Selection Systems for Black-box Numerical Optimization
Ryoji Tanabe
2022-04-27
Neural and Evolutionary Computing · Computer Science
Biobjective Performance Assessment with the COCO Platform
Dimo Brockhoff, Tea Tušar, Dejan Tušar, Tobias Wagner +2
2016-05-09
Neural and Evolutionary Computing · Computer Science
Towards a Theory-Guided Benchmarking Suite for Discrete Black-Box Optimization Heuristics: Profiling $(1+\lambda)$ EA Variants on OneMax and LeadingOnes
Carola Doerr, Furong Ye, Sander van Rijn, Hao Wang +1
2018-08-20
Machine Learning · Computer Science
On Constructing Algorithm Portfolios in Algorithm Selection for Computationally Expensive Black-box Optimization in the Fixed-budget Setting
Takushi Yoshikawa, Ryoji Tanabe
2024-05-21
Machine Learning · Statistics
Benchmarking Simulation-Based Inference
Jan-Matthis Lueckmann, Jan Boelts, David S. Greenberg, Pedro J. Gonçalves +1
2021-04-12
Machine Learning · Computer Science
OpenBox: A Python Toolkit for Generalized Black-box Optimization
Huaijun Jiang, Yu Shen, Yang Li, Beicheng Xu +4
2024-05-17
Artificial Intelligence · Computer Science
COCOpf: An Algorithm Portfolio Framework
Petr Baudiš
2014-05-16
Neural and Evolutionary Computing · Computer Science
A Complementarity Analysis of the COCO Benchmark Problems and Artificially Generated Problems
Urban Škvorc, Tome Eftimov, Peter Korošec
2021-04-28
Performance · Computer Science
Stochastic Superoptimization
Eric Schkufza, Rahul Sharma, Alex Aiken
2012-11-06
Machine Learning · Computer Science
ZOQO: Zero-Order Quantized Optimization
Noga Bar, Raja Giryes
2025-01-14
Performance · Computer Science
A Continuous Benchmarking Infrastructure for High-Performance Computing Applications
Christoph Alt, Martin Lanser, Jonas Plewinski, Atin Janki +4
2024-06-12
R2 v1 2026-06-22T13:20:33.334Z