It is known in Jaggi's paper that the traditional OMP algorithm for sparse optimization problems could fit into FW type algorithm. This generalized the usage of FW algorithm to unconstrained optimization problem with sparse solution. I would implement the variants of FW type algorithm, including the unconstrained case using different atom norms, and unify them into the same code structure.

Furthermore, it is easy to add regularization on atom norms for this unconstrained FW type algorithms. The benefit of doing this is twofold, Firstly, it can give faster convergence speed in optimization. Secondly, the freedom of adding different regularization parameters usually gives better results in applications. I will add this functionality in the code and compare the performance in these 2 aspects.

Moreover, I will try real world problems such as dictionary learning that will call this FW solver, to give a further demonstration and comparison of the usage of the solver I implemented.



Chenzhe Diao


  • stephentu