{"result":["QJ9zsb8n","gbBxNmxJ"],"entities":{"projects":[{"uid":"QJ9zsb8n","title":"Long Context Coding Evaluation Dataset for Gemini CLI","body":"This project builds a long context and complex reasoning coding evaluation dataset for Gemini CLI. It focuses on multi file, multi language software engineering tasks, difficulty certification, agentic evaluation, and structured failure analysis to better measure real coding agent performance on large repositories.","size":"medium","tech_tags":["Python, TypeScript, JavaScript, Go, Rust, Java, Do"],"topic_tags":["benchmarking, coding agents, long context, evaluat"],"status":"passed","phase":"final_eval_end","date_created":"2026-04-29T00:01:22.146624Z","date_updated":"2026-08-23T16:52:43.323769Z","assigned_mentors":["Akhilesh Kumar2"],"project_length":12,"github_username":null,"github_url":null,"contributor_id":"106584237898647208046","contributor_name":"Letian Li","body_short":"This project builds a long context and complex reasoning coding evaluation dataset for Gemini CLI. It focuses on multi file, multi language software...","proposal_id":"CjBSrozt","date_uploaded":"2026-03-31T16:58:55.956513Z","organization_slug":"gemini-cli","organization_name":"Gemini CLI","program_slug":"2026","milestones":[{"name":"project_work_start","datetime":"2026-05-25T18:00:00Z","phase":"work_period_1"},{"name":"midterm_evaluation_start","datetime":"2026-07-06T18:00:00Z","phase":"midterm_eval_accept"},{"name":"midterm_evaluation_deadline","datetime":"2026-07-10T18:00:00Z","phase":"work_period_2"},{"name":"final_submission_start","datetime":"2026-08-17T18:00:00Z","phase":"final_submission"},{"name":"final_submission_deadline","datetime":"2026-08-24T18:00:00Z","phase":"final_eval_accept"},{"name":"final_evaluation_start","datetime":"2026-08-24T18:00:00Z","phase":"final_eval_accept"},{"name":"final_evaluation_deadline","datetime":"2026-08-31T18:00:00Z","phase":"final_eval_end"},{"name":"project_completion","datetime":"2026-11-09T18:00:00Z","phase":"project_complete"}],"project_code_url":"https://github.com/li-letian/long-ctx-coding-evaluation-poc/tree/final-submission"},{"uid":"gbBxNmxJ","title":"Behavioral evals, Quality, and the OSS Community","body":"This proposal focuses on improving the behavioral evaluation (eval) infrastructure of Gemini CLI to ensure reliable and safe AI tool usage. The current system has significant gaps, including missing eval coverage for ~37.5% of tools, a large number of non-CI-blocking (flaky) tests, and lack of tooling for debugging and contributing to evals.\n\nTo address this, the proposal introduces a three-part approach:\n\nExpand and Stabilize Behavioral Evals\nWrite 15–20 new evals covering uncovered tools and missing behaviors (error recovery, multi-step reasoning, safety, ambiguity handling).\nStabilize flaky evals and promote reliable ones to CI-blocking status.\nFocus on testing tool-call decisions instead of text output for robustness.\nEnable Contributor Participation via Skills & Subagents\nIntroduce an eval-writer skill to help generate evals.\nBuild subagents like eval_investigator (debugging failures) and quality_analyst (coverage analysis).\nMake eval development accessible to open-source contributors.\nFix Ecosystem Gaps (Process, Tooling, Documentation)\nDefine clear eval review, promotion, and regression-handling processes.\nImprove logging with readable tool-call traces.\nCreate the Eval Development Kit (EDK) with tools for:\nEval inventory & coverage analysis\nValidation checks\nAuto-generating evals from real chat logs\nAdd documentation, tutorials, and GitHub automation for easier community contributions.\n\nOverall, the goal is to transform evals from a limited internal system into a scalable, contributor-friendly quality framework, enabling the OSS community to actively improve AI behavior, reliability, and performance.","size":"large","tech_tags":["typescript","Vitest"],"topic_tags":["cli","Agentic AI"],"status":"passed","phase":"final_eval_end","date_created":"2026-04-29T00:06:16.111208Z","date_updated":"2026-08-22T13:51:18.184466Z","assigned_mentors":["Christian Gunderman"],"project_length":12,"github_username":null,"github_url":null,"contributor_id":"102094167948979954424","contributor_name":"VedantxMahajan","body_short":"This proposal focuses on improving the behavioral evaluation (eval) infrastructure of Gemini CLI to ensure reliable and safe AI tool usage. The...","proposal_id":"zQK0ynTJ","date_uploaded":"2026-03-31T11:39:50.739115Z","organization_slug":"gemini-cli","organization_name":"Gemini CLI","program_slug":"2026","milestones":[{"name":"project_work_start","datetime":"2026-05-25T18:00:00Z","phase":"work_period_1"},{"name":"midterm_evaluation_start","datetime":"2026-07-06T18:00:00Z","phase":"midterm_eval_accept"},{"name":"midterm_evaluation_deadline","datetime":"2026-07-10T18:00:00Z","phase":"work_period_2"},{"name":"final_submission_start","datetime":"2026-08-17T18:00:00Z","phase":"final_submission"},{"name":"final_submission_deadline","datetime":"2026-08-24T18:00:00Z","phase":"final_eval_accept"},{"name":"final_evaluation_start","datetime":"2026-08-24T18:00:00Z","phase":"final_eval_accept"},{"name":"final_evaluation_deadline","datetime":"2026-08-31T18:00:00Z","phase":"final_eval_end"},{"name":"project_completion","datetime":"2026-11-09T18:00:00Z","phase":"project_complete"}],"project_code_url":"https://gist.github.com/ved015/f202faa874381037eac5cdeddd52a498"}],"organizations":[{"name":"Gemini CLI","slug":"gemini-cli","logo_url":"https://summerofcode.withgoogle.com/media/org/gemini-cli/mhi8cx7tyjt91i6n-360.png","website_url":"https://geminicli.com","tagline":"Gemini CLI brings the power of Gemini directly into your terminal","license":"Apache-2.0","categories":["Development tools","Artificial Intelligence"],"contributor_guidance_url":"https://github.com/google-gemini/gemini-cli/blob/main/CONTRIBUTING.md","description":"The Gemini CLI team maintains the Gemini CLI OSS project in order to deliver a high-quality, state-of-the-art agent to a large, global user base. Maintainers manage a set of public and internal (Google) backlog goals, open source community contributions, and build/release infrastructure.","tech_tags":["typescript","GenAI","MCP","Software Agent","A2A"],"topic_tags":["developer tools","GenAI"],"contact_links":[{"name":"email","value":"gemini-cli-soc-2026@google.com"},{"name":"twitter","value":"https://x.com/geminicli"}],"source_code":"https://github.com/google-gemini/gemini-cli/","ideas_link":"https://docs.google.com/document/d/1iaMZliqwUn-ACyZAbgzdXmDiQZ7l5gp8UQIIY2BnPO8/edit?usp=sharing","direct_comm_methods":[{"name":"email","value":"gemini-cli-soc-2026@google.com"}],"social_comm_methods":[{"name":"twitter","value":"https://x.com/geminicli"}]}],"programs":[{"name":"2026","slug":"2026","is_active":true,"phase":"project_work","terms_contributor":"2026.01","terms_member":"2026.01","terms_organization":"2026.01","terms_rules":"2026.01","date_created":"2025-11-24T21:40:30.291707Z","date_updated":"2026-05-18T16:59:18.670079Z","milestones":[{"name":"prog_announce","datetime":"2025-12-03T19:00:00Z","label":"Program Announce Date"},{"name":"org_application_start","datetime":"2026-01-19T18:00:00Z","label":"Org Applications Open"},{"name":"org_application_deadline","datetime":"2026-02-03T18:00:00Z","label":"Org Application Deadline"},{"name":"org_notification","datetime":"2026-02-18T18:00:00Z","label":"Org Notification Date"},{"name":"org_announce_public","datetime":"2026-02-19T18:00:00Z","label":"Accepted Orgs Announced"},{"name":"contributor_proposals_open","datetime":"2026-03-16T18:00:00Z","label":"Contributor Proposals Open"},{"name":"proposal_deadline","datetime":"2026-03-31T18:00:00Z","label":"Proposal Deadline"},{"name":"proposal_ranking_deadline","datetime":"2026-04-21T18:00:00Z","label":"Proposal Ranking Deadline"},{"name":"project_announce_oa","datetime":"2026-04-29T18:00:00Z","label":"Projects Announced to Orgs"},{"name":"project_announce_public","datetime":"2026-04-30T18:00:00Z","label":"Accepted Projects Announced"},{"name":"project_work_start","datetime":"2026-05-25T18:00:00Z","label":"Project Work Period Start"},{"name":"program_end","datetime":"2026-11-11T18:00:00Z","label":"Program End Date"},{"name":"midterm_evaluation_start","datetime":"2026-06-22T18:00:00Z","label":"Midterm Evaluation Start (small project)","project_size":"small"},{"name":"midterm_evaluation_deadline","datetime":"2026-06-26T18:00:00Z","label":"Midterm Evaluation Deadline (small project)","project_size":"small"},{"name":"final_submission_start","datetime":"2026-07-20T18:00:00Z","label":"Final Submission Start (small project)","project_size":"small"},{"name":"final_submission_deadline","datetime":"2026-07-27T18:00:00Z","label":"Final Submission Deadline (small project)","project_size":"small"},{"name":"final_evaluation_start","datetime":"2026-07-27T18:00:00Z","label":"Final Mentor Evaluation Start (small project)","project_size":"small"},{"name":"final_evaluation_deadline","datetime":"2026-08-03T18:00:00Z","label":"Final Mentor Evaluation Deadline (small project)","project_size":"small"},{"name":"project_completion","datetime":"2026-08-31T18:00:00Z","label":"Project Completion Date (small project)","project_size":"small"},{"name":"midterm_evaluation_start","datetime":"2026-07-06T18:00:00Z","label":"Midterm Evaluation Start (medium project)","project_size":"medium"},{"name":"midterm_evaluation_deadline","datetime":"2026-07-10T18:00:00Z","label":"Midterm Evaluation Deadline (medium project)","project_size":"medium"},{"name":"final_submission_start","datetime":"2026-08-17T18:00:00Z","label":"Final Submission Start (medium project)","project_size":"medium"},{"name":"final_submission_deadline","datetime":"2026-08-24T18:00:00Z","label":"Final Submission Deadline (medium project)","project_size":"medium"},{"name":"final_evaluation_start","datetime":"2026-08-24T18:00:00Z","label":"Final Evaluation Start (medium project)","project_size":"medium"},{"name":"final_evaluation_deadline","datetime":"2026-08-31T18:00:00Z","label":"Final Evaluation Deadline (medium project)","project_size":"medium"},{"name":"project_completion","datetime":"2026-11-09T18:00:00Z","label":"Project Completion Date (medium project)","project_size":"medium"},{"name":"midterm_evaluation_start","datetime":"2026-07-06T18:00:00Z","label":"Midterm Evaluation Start (large project)","project_size":"large"},{"name":"midterm_evaluation_deadline","datetime":"2026-07-10T18:00:00Z","label":"Midterm Evaluation Deadline (large project)","project_size":"large"},{"name":"final_submission_start","datetime":"2026-08-17T18:00:00Z","label":"Final Submission Start (large project)","project_size":"large"},{"name":"final_submission_deadline","datetime":"2026-08-24T18:00:00Z","label":"Final Submission Deadline (large project)","project_size":"large"},{"name":"final_evaluation_start","datetime":"2026-08-24T18:00:00Z","label":"Final Evaluation Start (large project)","project_size":"large"},{"name":"final_evaluation_deadline","datetime":"2026-08-31T18:00:00Z","label":"Final Evaluation Deadline (large project)","project_size":"large"},{"name":"project_completion","datetime":"2026-11-09T18:00:00Z","label":"Project Completion Date (large project)","project_size":"large"}],"stats":{"orgs_accepted":184,"allocated_projects":1156},"project_sizes":["small","medium","large"]}]},"meta":{}}