The multimedia search engine

vitrivr is an open source content-based multimedia retrieval stack that supports management of and searching in large, mixed media collections containing images, videos, audio files and 3D models. The software originates at the Databases and Information Systems (DBIS) research group of the University of Basel, Switzerland and has evolved into a thriving community involving people inside and outside of academia who’s common interest is multimedia retrieval.

The vitrivr-stack and individual components thereof have won various awards over the years. The stack consists of three independent systems written in Java, Kotlin and Typescript:

Cottontail DB: Is a column store that combines classical boolean retrieval and vector space retrieval used for multimedia search in a unified data and query model. It can be used as a standalone component independent of vitrivr and queries can be formulated through a gRPC API.

Cineast: Is a feature extraction and query engine, that generates feature descriptors - often vectors - stores them in the database layer or uses them for lookup. Furthermore, Cineast fuses results generated by different feature modules to produce the final output. Cineast can be interacted with through REST as well as gRPC.

vitrivr NG: Is our user-interface written in Angular / Typescript. It offers different means of query formulation, displays results and enables exploration of results and individual items.

Together, these three components make up a self-contained system for an end-to-end multimedia retrieval experience. In addition to supporting various media types, the stack also offers support for different query modes such as Query-by-Example, Query-by-Sketch, Text-based queries etc. The modular nature of the stack allows for easy extension on many different levels and integration with machine learning methods using completely different ecosystems.

lightbulb_outline View ideas list


  • java
  • kotlin
  • typescript
  • angular
  • tensorflow


comment IRC Channel
mail_outline Contact email

vitrivr 2021 Projects

  • Tanmay Singal
    Implement IIIF read in Cineast
    The International Image Interoperability Framework (IIIF) describes a standard to interact with images that are being adopted by an increasing number...
  • Karan Singh
    Vitrivr Reads
    Text in a video often conveys information that is not easily expressed otherwise. This project will implement and compare the various existing...