This project aims to optimize the performance of Nemo by dynamically splitting skewed tasks of a job based on the information collected from runtime metrics, executing them in parallel, and merging them afterwards. The goal is achieved through the following four steps: First, gather relevant metrics during execution. Second, using the gathered statistics, determine which task is skew(i.e. needs a longer time to finish) periodically. Third, Split the skewed tasks in half and allocate the latter splitted half to the new task. And lastly, execute the new task and check its performance.

Student

Hwarim Hyun

Mentors

  • Taegeon Um
  • Won Wook SONG
close

2021