Federating workflow execution using GA4GH Cloud APIs

The Task Execution Service (TES) API and the Workflow Execution Service (WES) are two of the several standards defined by the Global Alliance for Genomics and Health (GA4GH). The global picture is that TES and WES are designed to complement each other in running workflows by first identifying and scheduling the different individual tasks in the right order (WES) and then executing them in a containerized manner. This project is the implementation of a federated mechanism of task execution, i.e., different tasks of the same workflow will be executed in different Kubernetes clusters. These clusters can be in different geographical locations and be managed by different institutions. This can be achieved by introducing a dedicated “gateway TES” implementation, which receives an incoming task, runs a task distribution logic middleware to identify, out of several options, the most suitable TES instance, then relays the incoming task to that instance.