EvalAI is a platform to host and participate in AI challenges around the globe. To host a challenge, challenge creation is one of the core features which is utilized by challenge hosts to create AI challenges. The idea is to use private GitHub repositories to host the challenge files instead of the zip file. The next step is to build and integrate a continuous deployment pipeline with GitHub so that for every new commit in the challenge repository, the changes are automatically reflected on the UI. We will also build support for tests so that new commits are fully tested before they are pushed to the live challenge hosted on EvalAI. The goal is to enhance the challenge creation experience for challenge hosts and set up a challenge on EvalAI by the challenge hosts involving minimal human effort from the EvalAI team.