Live boardHow it works ICML 2026 resultsFor judgesEarlier runsGitHub
CodeRabbit for research papers

Your paper says it. Does your code?

Send us your paper and its code. The PaperCI agent reads it, talks with you about what matters, rebuilds your environment from scratch in a locked-down sandbox, re-runs your experiments, and checks every headline claim against what your code actually produces, before reviewers or readers do.

  • An agent, not a form: it asks you only what it needs (data mirror, entry point, which claims matter) and comes back to you when it's stuck
  • Claims are sealed first: the engineer agent never sees the numbers it has to hit
  • Verified by a different AI model family, three reruns, network switched off
  • Catches shortcuts: hardcoded answers, copied or replayed results, scripts that skip your code
  • Private claim-by-claim review, a self-contained environment anyone can replay, and re-verification on every new commit

Verify my paper

Private by default. Usually 10-20 minutes for CPU-scale code.
Today: CPU-scale code, no GPU, no paid API keys. GPU and simulator workloads are on the roadmap.

What the agent does with your paper

1
Reads and sealsExtracts your headline claims with verbatim quotes and seals them (sha256). The engineer agent only sees metric names.
2
Talks with youShares what it found and its plan, asks the 1-3 questions that change the plan, and works with your answers.
3
Rebuilds and verifiesFresh non-root container, dependencies and data from GitHub, PyPI, Hugging Face, Zenodo. Then three offline reruns judged by a different model family.
4
Reviews and keeps watchingClaim-by-claim review, fixes, badge and a replayable environment. Asks you when it's stuck; re-verifies on new commits.
PaperCI · built for neatHack 2026 · traces by neatlogs · sessions by Entire