Call for Artifacts

All papers accepted at ACM CAIS’26 are encouraged to participate in the artifact evaluation process.

Artifacts must be consistent with the paper, as complete as possible, reasonably well documented, and easy to reuse. These goals are reflected in the three badges that can be awarded to each paper: Available, Functional, and Results Reproduced (or Reproduced, for short). The purpose of the AEC is to help authors meet these goals and to award badges to artifacts that satisfy the criteria. Note that for ACM CAIS’26, the AE is single-blind (see below).

Questions about artifact evaluation can be directed to aec-chairs@caisconf.org.

To be considered in the AE process, at least one contact author for the submission must be reachable and respond to questions in a timely manner during the evaluation period, allowing sufficient back-and-forth between the AEC and the authors. Please check the AEC timeline and important dates here.

Registration and Submission

Link to HotCRP portal: https://acm-cais26-ae.hotcrp.com/

Please submit your artifacts to the AE HotCRP portal by providing an URL or a packaged artifact, selecting which artifact badges you apply for, and provide an artifact appendix that describes the artifact.

The effort that you put into packaging your artifacts has a direct impact on the committee’s ability to make well-informed decisions. Please package your artifacts with care to make it as straightforward and easy as possible for the AEC to understand and evaluate their quality.

Note: If you need permission from your organization’s legal or IT department to publish your artifact or give evaluators access to custom hardware, submit that request as soon as possible, otherwise evaluators may not have sufficient time to audit your artifact.

Process

Authors are invited to submit artifacts shortly after their papers are accepted. Because the time between paper acceptance and artifact submission is short, the AEC Chairs encourage authors to begin preparing their artifacts while their papers are still under review.

At the time of artifact submission, authors choose which badges they want to pursue. Please read instructions and criteria, here.

After the artifact submission deadline, evaluators will review each artifact using the corresponding paper and artifact appendix as guides. Evaluators may communicate with authors exclusively through HotCRP (to preserve anonymity) to resolve minor issues and ask clarifying questions throughout the evaluation process. Evaluation begins with a “kick-the-tires” period, during which evaluators confirm that they can access their assigned artifacts and perform basic operations, such as compiling and running a minimal working example. Artifact evaluations include feedback, allowing authors to improve both their artifacts and their papers based on that feedback.

Artifact Details

Artifacts can include software, datasets, survey results, test suites, mechanized proofs, and similar materials. Pen-and-paper proofs are not accepted, as evaluators often lack the time and expertise to review them carefully. Physical objects, such as computer hardware, also cannot be accepted because they are difficult to make available to evaluators. To the extent possible, artifacts should be able to run on commodity hardware (e.g., laptop or desktop systems). If this requirement cannot be met, please contact the AEC Chairs in advance so that arrangements can be made for evaluators to access special hardware (e.g., your own). More detailed artifact packaging instructions are available here.

When submitting your artifact, please specify which combination of the three badges you are applying for. For the Functional and Reproduced badges, AEC members will attempt to use your artifact to run the experiments described in your paper.

Submitting an artifact for evaluation does not give the AEC permission to make its contents public or to retain any part of it after the evaluation. Thus, authors are free to include proprietary models, data files, or code in their artifacts. Participating in artifact evaluation does not require public release of the artifact, though public release is highly encouraged.

AEC members may contact authors during the evaluation period, for example, to ask for help if they are unable to get the artifact to work, and authors are expected to respond to such requests. However, your goal as an author should be to present and document your artifact so that AEC members can use it and complete the evaluation successfully on their own (ideally without needing to interact with the authors). To ensure that your instructions are complete, we recommend testing them on a fresh setup before submission, following exactly the instructions you provide.

Reviewing and Anonymity

Artifact evaluation is “single-blind”, meaning that the identities of authors will be known to reviewers, but authors will not know which Artifact Evaluation Committee (AEC) members reviewed their artifacts.

To maintain reviewer anonymity, authors should not embed analytics or other tracking tools in the websites hosting their artifacts for the duration of the artifact evaluation period. This helps maintain reviewer confidentiality. In cases where tracking is unavoidable, authors should notify the AEC Chairs in advance so that AEC members can take adequate precautions