We are looking for members of the Artifact Evaluation Committee (AEC), who will contribute to SOSP’23 Artifact Evaluation (AE) process by reviewing submitted artifacts. AEC membership is especially suitable for researchers early in their career, such as PhD students. Even as a first-year PhD student, you are welcome to join the AEC, provided you are working in a topic area covered by SOSP (such as operating systems, file and storage systems, distributed systems, cloud computing, and other areas of computer systems research). You can be located anywhere in the world as all committee discussions will happen online.
As an AEC member, you will not only help promote the reproducibility of experimental results in systems research, but also get to familiarize yourself with research papers just accepted for publication at SOSP 2023 and explore their artifacts. For a given artifact, you may be asked to evaluate its public availability, functionality, and/or ability to reproduce the results from the paper. You will be able to discuss with other AEC members and anonymously interact with the authors as necessary, for instance if you are unable to get the artifact to work as expected. Finally, you will provide a review for the artifact to give constructive feedback to its authors, discuss the artifact with fellow reviewers, and help award the paper artifact evaluation badges.
We expect that each member will evaluate 2-4 artifacts and that each evaluation will take around 10–20 hours. AEC members are expected to allocate time to choose the artifacts they want to review, to read the chosen papers, to evaluate and review the corresponding artifacts, and to be available for online discussion until artifact notification deadline. Please ensure that you have sufficient time and availability for the AEC during the AE periods July 31 to September 1 2023. Please also ensure you will be able to carry out the evaluation independently, without sharing artifacts or related information with others and limiting all the discussions to within the AEC.
We expect that evaluations can be done on your own computer (any moderately recent desktop or laptop computer will do). In other cases and to the extent possible, authors will arrange their artifacts so as to run in community research testbeds or will provide remote access to their systems (e.g., via SSH) with proper anonymization.
How to Apply
If you are interested in taking part in the AEC, please complete this online self-nomination form link here.
Deadline: July 6, 2023, Anywhere on Earth
You can contact the AEC chairs for any questions.