It has been shown that only a small portion of all the files in a container image is necessary to run the image itself. This is even more accentuated in scientific container images since they usually include complex software stacks comprising hundreds of thousands of files, and often not all the files are needed for each task. CernVM-FS is a globally-distributed file system used to distribute software to data centers and end-user workstations in a fast, scalable and reliable way. Files and file metadata are downloaded on demand and aggressively cached. Hence we can utilize the lazy load capabilities of CVMFS with various container engines to get the containers running faster than usual. There is already a plugin for Docker available for this purpose and a prototype for Kubernetes is also being developed. This project aims to utilize CVMFS capabilities in podman workflow, to quickly load big scientific container images while maintaining the isolation and convenience of containers.

Organization

Student

Mohit Tyagi

Mentors

  • Simone Mosciatti
  • Giuseppe Scrivano
close

2020