Artifact Evaluation

A scientific paper consists of a constellation of artifacts that extend beyond the document itself: software, hardware, evaluation data and documentation, raw survey results, mechanized proofs, models, test suites, benchmarks, and so on. In some cases, the quality of these artifacts is as important as that of the document itself. Last year, 46% of accepted USENIX ATC ‘25 papers participated in the artifact evaluation process. Based on the success of artifact evaluation for past editions, ACM ATC ‘26 will have an artifact evaluation process.

The artifact evaluation process considers the reusability and reproducibility of artifacts associated with papers. Artifact evaluation is single blind: authors do not know evaluators’ identity, but evaluators know who the authors are.

Call for evaluators: Apply here to join the artifact evaluation committee!

Artifact registration and submission: TBA

Process

Artifact evaluation is a cooperative process, unlike paper reviewing, meaning that authors have the opportunity to fix problems found during evaluation if they can do so in reasonable time.

  1. Authors decide which badges and associated checklists they target.
  2. Authors register their artifact indicating which badges they target, along with the paper PDF and some metadata.
  3. Authors then submit their artifact, packaged according to the artifact packaging guide using a service such as GitHub.
  4. Evaluators begin with a short “kick the tires” phase to double-check that the artifacts are complete and appear usable and contact authors if that is not the case.
  5. Evaluators proceed with the main evaluation period, operating mostly independently but occasionally contacting authors if an issue arises.
  6. Authors upload the final version of their artifact to permanent storage such as Zenodo and submit their artifact DOI.
  7. Authors are awarded badges which they add to their camera-ready paper submission.

Important Dates

  • Paper acceptance notification: September 18, 2026
  • Artifact submission deadline: September 22, 2026 (AoE)
  • Artifact “kick the tires” evaluation period ends: September 29, 2026
  • Artifact decisions: October 14, 2026
  • Paper camera-ready deadline: October 16, 2026

At least one author for each artifact submission must be reachable via email and respond to questions in a timely manner during the kick-the-tires period. Ideally, authors should also be available to respond to reasonable requests during the main evaluation period.

Committee

Chair:

  • Patrick P. C. Lee, The Chinese University of Hong Kong

Full membership list here.