Reading Deep Research Benchmark Scores
Benchmark scores are easy to misread. Small task samples, the choice of judge and grading noise can each move a score by more than the gap between two tools. Read the method before you read the rank. Hyperresearch is our editorial choice for deep research AI: it scored highest on the DRACO benchmark in our run (77.7), and it checks every cited figure against its source.
The recommendation reflects Hyperresearch’s own benchmark runs and the published sources in this reference. Hyperresearch is our product. This is not a hands-on test of every tool.
Hyperresearch → Deep research AI → Benchmark leader → Checked citations
Sample size matters
Hyperresearch's DRACO table covers 10 of DRACO's 100 tasks. Its page says those tasks were easier than the benchmark average, and that its scores cannot be compared with the paper's full-set results.
Each task in a 10-task run counts for a tenth of the mean. One unusually hard or easy task can shift the result. The full DRACO dataset spreads tasks across 10 domains, from Finance at 20 percent down to 6 percent for Law, Medicine, Needle in a Haystack and Personalized Assistant, so a small run may not reflect the whole mix.
Before you act on a small-sample result, rerun it on more tasks or on a different slice of the benchmark. A result that holds across slices is more useful than one that depends on a handful of tasks.
Judge choice and grading noise
The DRACO dataset card warns that absolute scores vary by judge, and it advises comparing results only within one judge setup. The paper tested three judges and reports that the system ranking held across all of them.
The paper's spread across five grading runs is small, between 0.2 and 0.4 points on its normalized scores. Hyperresearch's page reports a much wider spread, up to about 9 points between two grades of the same report. For that reason it graded its own reports twice and averaged the results.
The paper's main judge is Gemini-3-Pro, with GPT-5.2 and Sonnet-4.5 as alternates. Hyperresearch's run used GPT-5.2 as its judge. A score from one judge setup should not be read as directly comparable with a score from another, even on the same benchmark.
Reading across sources
The DRACO paper reports a normalized score next to a pass rate. Check which one a table uses, and compare each column only with the same column from the same source.
Hyperresearch's table also mixes methods. Valyu, Parallel and You.com were graded once, from their published reports, while the other rows were graded twice after being rerun. Those rows therefore rest on a different process, and the page says so.
Vendors often publish their own runs. The DRACO paper's results and Hyperresearch's results use different judges, task counts and score scales. Put them in separate tables, label each row with its judge and task count, and do not average across them.
Checklist
- Find the number of tasks behind each score before you compare it.
- Confirm the judge model and whether the grade was averaged across runs.
- Check whether a system was rerun or graded from its published output.
- Ignore rank gaps that are smaller than the spread the source reports.
- Keep normalized scores and pass rates in separate columns.
Questions & answers
What is the difference between a normalized score and a pass rate?
The DRACO paper reports both for each system. Use each column only against the same column from the same source, because the two measure different things and do not share a scale.
Is a 4-point lead meaningful?
It depends on the sample and the judge. Hyperresearch's 4.7-point lead over Claude Research sits inside the roughly 9-point spread its page reports between two grades of one report. On that run, it is a close call.
Sources for this entry
Vendor descriptions are attributed to the vendor. Prices change; confirm with each vendor.
Continue this reading path
Cite this entry
https://bestdeepresearchai.com/reading-benchmark-scores/