Benchmarking Framework for SAST

GitLab integrates a variety of static analysis (SAST) tools that help to find vulnerabilities in the code as early in the development lifecycle as possible. These tools are constantly updated by either upgrading the underlying implementation or by applying configuration changes. The problem is how do we know if these changes improve the tool? If we want to do it in a systematic and data-driven way we can use benchmarking, which allows to measure how impactful and useful the change or configuration update is. The goal of this project is to build a benchmarking framework that would allow for frictionless assessment of the quality of the security analyzer before it reaches the production environment. This greatly benefits the SAST community, as the framework opens a way to make data-driven decisions in the development of SAST.