CMA-ES - Performance in Practice

Performance in Practice

In contrast to most other evolutionary algorithms, the CMA-ES is, from the users perspective, quasi parameter-free. However, the number of candidate samples λ (population size) can be adjusted by the user in order to change the characteristic search behavior (see above). CMA-ES has been empirically successful in hundreds of applications and is considered to be useful in particular on non-convex, non-separable, ill-conditioned, multi-modal or noisy objective functions. The search space dimension ranges typically between two and a few hundred. Assuming a black-box optimization scenario, where gradients are not available (or not useful) and function evaluations are the only considered cost of search, the CMA-ES method is likely to be outperformed by other methods in the following conditions:

  • on low-dimensional functions, say, for example by the downhill simplex method or surrogate-based methods (like kriging with expected improvement);
  • on separable functions without or with only negligible dependencies between the design variables in particular in the case of multi-modality or large dimension, for example by differential evolution;
  • on (nearly) convex-quadratic functions with low or moderate condition number of the Hessian matrix, where BFGS or NEWUOA are typically ten times faster;
  • on functions that can already be solved with a comparatively small number of function evaluations, say no more than, where CMA-ES is often slower than, for example, NEWUOA or Multilevel Coordinate Search (MCS ).

On separable functions the performance disadvantage is likely to be most significant, in that CMA-ES might not be able to find at all comparable solutions. On the other hand, on non-separable functions that are ill-conditioned or rugged or can only be solved with more than function evaluations, the CMA-ES shows most often superior performance.

Read more about this topic:  CMA-ES

Famous quotes containing the words performance and/or practice:

    “So long as the source of our identity is external—vested in how others judge our performance at work, or how others judge our children’s performance, or how much money we make—we will find ourselves hopelessly flawed, forever short of the ideal.”
    —Melinda M. Marshall (20th century)

    “Certainly, young children can begin to practice making letters and numbers and solving problems, but this should be done without workbooks. Young children need to learn initiative, autonomy, industry, and competence before they learn that answers can be right or wrong.”
    —David Elkind (20th century)