In GSoC, I propose to introduce deep learning annotators in RoboSherlock for more various task. Because many deep learning frameworks and repositories are written in Python, I first improve RoboSherlock system to be able to run annotators written in Python. Then, I implement annotators using deep learning such as object detection, semantic segmentation, instance segmentation and so on into RoboSherlock system. As the final result of the project, the real robot execute a manipulation task with the deep-learning annotators. The main contributions of my proposal will be as belows:

  • Improve Robosherlock to execute annotators written in Python
  • Introduce several useful deep-learning annotators in Robosherlock
  • Implement the new annotators in daily-life task with real robot


Shingo Kitagawa


  • Alexis Maldonado
  • Ferenc Balint-Benczedi