Artifact Evaluation

EuroSys 2024 is the fourth edition of the EuroSys conference to include an Artifact Evaluation (AE) process, allowing authors to submit an artifact for the research work discussed in the paper.

Scientific papers typically consist of artifacts that extend beyond the paper: software, hardware, data and documentation, raw survey results, mechanized proofs, models, test suites, benchmarks, and so on. The AE process promotes the reproducibility of experimental results of a scientific paper.

The dissemination of artifacts can benefit science as a whole. First and foremost, it encourages replication and reproduction, enabling scientists to extend previous work and solve questions about cases not considered by the original authors. For authors, publishing an artifact can impact how easily peers can build on it, use it as a comparison point, and propose collaborations.

The AE process of EuroSys recognizes authors who invest effort to make their work reusable and reproducible by others, helping the community quickly validate and compare alternative approaches. This includes authors’ efforts to make their artifact publicly available, documenting and packaging their work in a way that facilitates reuse, and structuring experiments such that they can be repeated and the results reproduced by other researchers.

AE is optional at EuroSys 2024. Authors of accepted papers have the option of submitting their artifacts shortly after the notification of their acceptance, although authors are encouraged to prepare their artifacts in advance. Each artifact is reviewed by the Artifact Evaluation Committee (AEC) between the author notification and the camera-ready deadline. A paper whose artifacts have passed the artifact evaluation process includes an appendix that details the artifacts and is recognized by one or more badges that appear on the paper’s first page.