Organizations: MLpack

Proposal ideas:

Implementing enhancement modules to improve the ability of original CMA-ES in terms of scalability, performance and finding global optimum.

Objectives:

  1. Implement 4 extension algorithms (restart-CMA-ES, active-CMA-ES, Sep-CMA-ES and Cholesky-CMA-ES) and get merged into the codebase ensmallen.
  2. Refactorize the folder ensmallen/include/ensmallen_bits/cmaes with proper APIs for different variations of CMA-ES.
  3. Write intensive test cases with different kinds of functions and sanity checks for each variation algorithm.
  4. Detail documentation on the code and create a real-world application of CMA-ES tutorial.

Table of contents:

  1. About Project
    1. Reason
    2. Summary of CMA-ES
    3. CMA-ES enhancement
    4. API Design
    5. Testing and Benchmarking
    6. Documentation
  2. Timeline

2 About Project

2.1 Reason

The reasons why I chose this project as GSoC project. I have been practicing C++ meta programming and multi-thread lately, mainly by reading "The C++ Programming Language" by the "father" of C++ Bjarne Stroustrup. I tried to implement the C++ matrix library from scratch recently, but everything seems overwhelming to me, so I decided to look for an open-sourced C++ project that aligns with my academic interests which are Statistics and Machine Learning. My mathematics is accumulated over the years, but yet I still have not tried to apply any related algorithm using my favorite programming language - C++

And surprisedly, mlpack GSoC idea on MCA-ES Enhancement is an intersection of my interests.

2.2 Summary of MCA-ES algorithm and changes to original algorithm recent years

I did some research into this field and found some new promising improvements which can easily be integrated into ensmallen library

So the MCA-ES falls into the category of Evolutionary Algorithm(EA) - Evolution strategy. EA algorithms find the candidate solutions in any type of optimization problem by constantly changing the original (randomly generated) population - an inspired biological evolution process.

A typical ES consists of 4 steps and repeats the steps until the desired population is generated - qualified by objective function(s)