Software-engineering benchmark
Resolving real software issues in code repositories.
Unit: % resolved · higher is better
No dated results yet.
Research · Agentic tracker
What agents can reliably do, from dated results.
An ongoing stream. Each series follows one benchmark over time; every point links to the primary source that reported it.
First entries arriving. Five benchmark series are defined, covering coding, computer use, task length, tool use and web research. We are collecting the first sourced entries for this view. Each one will link to the page it came from, and nothing here is a finding until it does.
Computer-use benchmark
Completing tasks by operating a real desktop or browser.
Unit: % completed · higher is better
No dated results yet.
Task-horizon measure
The length of task, in human time, an agent completes at a stated success rate.
Unit: minutes · higher is better
No dated results yet.
Tool-use benchmark
Following policy and calling the right tools across a multi-step customer-service style task.
Unit: % success · higher is better
No dated results yet.
Web-research benchmark
Finding and combining information across the open web.
Unit: % correct · higher is better
No dated results yet.
Scores are only comparable within one benchmark, one version and one reported setting. We note the setting when a source states it, and never merge different versions into one line.