Evaluation Infrastructure Optimization

This project aims to enhance EvalAI's functionalities by automating large worker deployments in AWS, adding relevant features for efficient challenge management and also writing a robust and efficient test suite. The focus of the project is: To automate large worker deployment processes on EvalAI using AWS EC2 instances, reducing infrastructure running costs. Make challenge management seamless and less reliant on the admins reducing the dependency of challenge hosts on EvalAI admins. To make EvalAI more reliable and error-free by incorporating tests for different frontend and backend components. Having robust tests prevents making code-breaking changes to the codebase. This task will include adding unit tests for the API suite, prediction upload evaluation workers, code upload evaluation workers (on EKS), and integration tests for end-to-end testing of all components.