This project aims to optimize Apache Nemo I/O with two main approaches:

  • Providing I/O using high-performance networking and storage hardware such as RDMA and NVMe by integrating Apache Crail to Nemo and
  • Utilizing off-heap memory for storing serialized data (e.g. intermediate data) to alleviate JVM garbage collection overhead and use zero-copy for transferring these off-heap data between workers to reduce unnecessary data copies.

Student

Haeyoon Cho

Mentors

  • Gyewon Lee
  • Jangho Seo
close

2019