Skip to content

Research

Studies of coding agents on real C and C++ code: what they get right, what they cost, and how we measured it.

  1. ByteAsk: Build-Aware Context and a Checking Harness for C++ Coding Agents

    Paper · Preprint

    ByteAsk is a C++ coding agent that uses the project's build settings to find the relevant code, gives the AI model that context before it starts, and builds and runs only the tests a change reaches. On the same AI model as Codex CLI, with the same number of runs fixed, it used 63–80% fewer tokens and cost 59–70% less.

    Pratyush Saini, CTO, ByteAsk Research; Anirudha Kulkarni, CEO, ByteAsk Research; Aman Gupta, Carnegie Mellon University

  2. Beyond Text Search: Context Engines for C++

    Paper · Preprint

    ByteAsk Context is a context engine for C and C++ coding agents that understands a codebase the way the project's own build does. On real issues from open-source C and C++ projects, it put a file that needed changing first about twice as often as sgrep and almost three times as often as Probe.

    Pratyush Saini, CTO, ByteAsk Research; Anirudha Kulkarni, CEO, ByteAsk Research; Aman Gupta, Carnegie Mellon University

  3. Build-Aware Context for Large C++ Codebases

    Paper · Preprint

    ByteAsk Context understands C and C++ code the way the project's build does and tells a coding agent where a change belongs. On one machine it indexed the Linux kernel's build in 9.3 minutes and LLVM's in 13.7, re-indexed a source-file edit in 5.8 seconds, and ranked a changed file first on real open-source issues about twice as often as sgrep.

    Pratyush Saini, CTO, ByteAsk Research; Anirudha Kulkarni, CEO, ByteAsk Research; Aman Gupta, Carnegie Mellon University

  4. Claude Code with a Language Server: Higher Cost, No Clear Gain in C/C++ Bug Fixes

    Comparing Claude Code with and without the clangd code navigation tool on 255 issues from 40 software projects.

    Paper · Preprint

    We tested whether a tool that helps Claude Code find its way around code would improve bug fixes. Across 255 C and C++ issues, the tested setup cost 11% more overall, with no clear improvement in successful fixes.

    Pratyush Saini, CTO, ByteAsk Research; Anirudha Kulkarni, CEO, ByteAsk Research