Benchmark
FlowBench
Can a tool follow every entry point of a real codebase, call by call, to the end? We ran vivibe on 25 well-known open-source projects to find out.
100%Trace completenessof 6,258 entry points
25/25Projects with no broken flow5 languages, pinned commits
71%Of entry points in FlowBench Verified7 hand-audited projects, 0 open wrong calls
- Wrong calls found21
- Fixed and locked by a test21
- Still open0
Every wrong call found was fixed in the analyzer and kept fixed by a golden test.
Results by project
From reference apps with 4 entry points to platforms with 3,299.
#LanguageCompletenessAudit
25 of 25 projects. Names open the code at the exact commit we read.
Methodology
- Task
- For every entry point a project has (an HTTP route, job, consumer, command or page), follow the flow call by call until it ends in code or a known resource such as a database or queue.
- Trace completeness
- The share of entry points whose flow is whole. A flow fails if it stops at a gap or at a call that cannot be resolved; guessing an edge is not allowed.
- Dataset
- 25 open-source projects in 5 languages, each read at a pinned commit. A project with even one broken flow is left out of FlowBench, so the set shows where vivibe is whole today, not every project tried.
- FlowBench Verified
- The 7 projects where people sampled call edges and read them against the source, over 17 rounds. Each wrong edge became a fix and a golden test.
- Setup
- vivibe-app ce5fab0e, 8 Nov 2026. The analysis is deterministic (tree-sitter, plus libclang and Roslyn); no model is involved, so a run gives the same result every time.
- Hand audits ran on analyzer versions 0.5 to 0.8, before 26 Oct 2026; the trace run used that day's build.
- Analysis time is not reported: the bench machine was overloaded during the run.
- Projects still being measured on the run date are not listed until they have a result.
Try it on your own code.
Open any repo on your Mac, or compare a branch, PR or working tree. vivibe is in private beta: ask to join, and sign in with the same email once you are in.