Incorporating a Large Language Model in Ganga to assist users
- Mentors
- Alexander Richards, Ulrik, Mark Smith
- Organization
- CERN-HSF
- Technologies
- python, vLLM
- Topics
- web, machine learning, LLM
Ganga is a tool for composing, running, and tracking computing jobs across a variety of backends and application types. Ganga primarily runs as a command-line tool and has an IPython-like prompt. My proposal lays out a plan to add a large language model to the Ganga prompt. It discusses options like cloud serving, local serving, and the challenges that come with each. It also addresses whether a rag system would make sense for the project at this juncture. Finally, I suggest a timeline breakup where I first focus on implementing a local solution followed by a cloud one.