This project aims to implement Node Labeling and Preemption API, which are existing feature of resource managers, on REEF. Node Labels allow users to group resources with similar characteristics or with similar ACL policies and Preemption is a way to respect elasticity and SLAs together.
Even though YARN and Mesos, the resource managers have existing features that supports Node Labeling and Preemption, REEF doesn’t have APIs that makes these features available. To make developers can exploit these features on REEF regardless of which resource manager they use, APIs that make available Node Labeling and Preemption will be implemented on REEF in this project.
- Support for the Node Labeling API as it exists in YARN and Mesos in REEF
- Support for the Preemption API as it exists in YARN and Mesos in REEF
Benefits for REEF
When Node Labeling and Preemption has been done in REEF, developer can exploit these features without directly dealing with resource managers such as YARN or Mesos, but through the same interface on REEF. It will increase portability and simplifies development of applications for cluster resource managers.