MediaPipe fills the gap of handling the implementation of models for different systems and allows the developers to focus more on experimentation than on system architectures. WebAssembly has become prominent in recent years as it can port C++ applications onto web browsers, making code more portable and utilizing the efficiencies of C++. In my GSoC internship, I will combine MediaPipe and WebAssembly by building an open-source realtime web video effects app. I will create a complete workflow of wrapping models in MediaPipe to be used in a webapp for running different ML-based video effects live in the browser using WebAssembly. The workflow will include wrapping the model binaries in calculators, create calculator graphs to use the model binaries, set up Bazel for WebAssembly compilation, write Emscripten bindings to access the models, and create a Typescript webapp to demonstrate how to access the WebAssembly binaries in realtime. I have extensive experience in C++ development, along with web development using Javascript. I have also been using MediaPipe for almost a year and am accustomed to the codebase. Hence, I am confident that I will be successful in my internship.

Organization

Student

Pinku Deb Nath

Mentors

  • Tyler Mullen
  • Michael Hays
close

2021