Contributor
Aryabhatta Dey

Incorporating a Large Language Model in Ganga to assist users


Mentors
Alexander Richards, Ulrik, Mark Smith
Organization
CERN-HSF
Technologies
python, vLLM
Topics
web, machine learning, LLM
Ganga is a tool for composing, running, and tracking computing jobs across a variety of backends and application types. Ganga primarily runs as a command-line tool and has an IPython-like prompt. My proposal lays out a plan to add a large language model to the Ganga prompt. It discusses options like cloud serving, local serving, and the challenges that come with each. It also addresses whether a rag system would make sense for the project at this juncture. Finally, I suggest a timeline breakup where I first focus on implementing a local solution followed by a cloud one.