Enhancing DBpedia with image-based querying

Currently, users can query DBpedia using text. Although text as an input is an efficient approach to query the graph, there are cases where we do not know what we are seeing. How does one search the knowledge graph (KG) in such cases? Imagine being able to query the DBpedia Knowledge Graph (DB-KG) using images! The idea here is to create a framework that can combine existing computer vision techniques with knowledge graphs. Doing this will enable us to query the existing knowledge graphs using multiple modalities: images and text. To this end, in this proposal, we examine and explore two aspects of DB-KG: (a) A framework to create an image-based KG out of existing DBpedia entries; (b) Using the graph created to perform tasks like image querying, text + image search, and using relevant input images to add more images to existing articles.