arXiv (Cornell University) · 2016 · 49 citations · 16 references
ComputingComputer ScienceAlgorithmsOptimizationPerformance MeasurementAlgorithm AnalysisAlgorithmic PerformanceBenchmarkingMetaheuristic OptimizationPerformance OptimizationSimulation-based OptimizationBenchmarking Of MetaheuristicsBlack-box TestingEmpirical DistributionEmpirical Distribution FunctionsObjective Function FormulationOptimization AlgorithmsPerformance BenchmarkingQuantitative Result Interpretation
We present an any-time performance assessment for benchmarking numerical\noptimization algorithms in a black-box scenario, applied within the COCO\nbenchmarking platform. The performance assessment is based on runtimes measured\nin number of objective function evaluations to reach one or several quality\nindicator target values. We argue that runtime is the only available measure\nwith a generic, meaningful, and quantitative interpretation. We discuss the\nchoice of the target values, runlength-based targets, and the aggregation of\nresults by using simulated restarts, averages, and empirical distribution\nfunctions.\n
16
On the Theory of Scales of Measurement
S. S. Stevens · Science · 1946 · 4.7K citations
An Introduction to the Bootstrap.
S. T. Buckland, B. Efron, Ryan J. Tibshirani · Biometrics · 1994 · 3.2K citations