We're happy to invite everyone to Fernanda Takahashi's Ph.D. qualification exam. Her work Sample Size Estimation For Power And Accuracy In The Experimental Comparison of Metaheuristics will be presented on May 31st 2017 2 p.m., at the Seminar Room T005. Besides Fernanda's advisor, ORCS Lab's Prof. Felipe Campelo, the qualification committee will be composed by:
- Prof. Thiago Noronha, DCC-UFMG
- Prof. Luiz Duczmal, Dep. Statistics - UFMG
- Prof. Helio Barbosa, LNCC
Click below for the abstract.
Update: click here for pictures!
Abstract: Experimental algorithmics contemplates the study of guidelines and methods for computational evaluation of algorithms. In the optimization field, it is useful for testing the performance of algorithms when solving a certain type of problem. In this work we are developing a methodology for generating adequate experimental designs for comparing the performance of optimization metaheuristics, with a focus on statistical power and accuracy in parameter estimation. In particular, we deal with sample size estimation for experiments involving optimization algorithms, both in terms of within-instance repeated executions and the number of instances required. The estimate of the sample size influences the power and accuracy of the tests performed, which means that the number of samples is directly related to the strength of conclusions that can be drawn from the experiment. An initial method for estimating the number of replications for a pair of algorithms on a given problem instance has been developed in the progress of this work, as well as some preliminary estimations of the number of instances needed for specific comparisons. More sophisticated techniques are to be developed and implemented, with the incorporation of sequential designs and more robust statistical methods.