Artifact Review Summary: Automating SmartNIC Offloading Insights for Network Functions

Artifact Details

Badges Awarded

Artifact Available Artifact Functional
Artifacts Available (v1.1) Artifacts Evaluated - Functional (v1.1)

Description of the Artifact

The artifact, Clara, is an automated tool to generate high-accuracy insights of offloading NF (network functions) from x86 server CPU to SmartNIC (i.e. Netronome). The artifact includes nine modules used in the paper, among which there are primarily six modules for generating offloading insights. The compute_prediction module is to transform unported NF to LLVM intermediate representation using program analysis and predict instructions generated by the compiler. The modules of algo_id, scale_out, ilp_solving, variable_reorder and nf_colocation are to generate porting strategies from multiple different perspectives, including acceleration, multicore scale-out, NF state placement, memory access coalescing and NF colocation.

To set up the environment to run the artifact, either follow README.md or use an author-provided vagrant virtual machine with all the dependencies installed (including clara.box and Vagrantfile). To start the VM, simply run vagrant up then vagrant ssh. The password is vagrant.

The authors also provided three demo videos about how to generate dataset for three modules of Clara, including compute_prediction, nf_colocation and scale_out.

Environment(s) Used for Testing

Below is a summary of different environments evaluated by the reviewers.

Environment 1

  • Intel(R) Xeon(R) CPU D-1548 with 16 (logical) cores and 64GB memory
  • Ubuntu 18.04
  • (Optional) Vagrant 2.2.18 and VirtualBox 6.1 (with 28 GB storage space needed at least)

Environment 2

  • Intel Core i9-10900
  • Ubuntu 20.04

Environment 3

  • Intel Xeon W-2133 CPU @ 3.60GHz
  • Ubuntu 18.04

Step-By-Step Instructions to Exercise the Artifact

  1. Install the dependencies and build and install click, following README.MD in the root directory of the artifact.
  2. Enter the compute_prediction folder and then run the scripts to predict NF performance parameters by using different ML algorithms and compare their accuracies, following the corresponding README.MD.
  3. Enter the algo_id folder, and then run the scripts to generate insights and lookup and compare accuracies of using different ML algorithms, following the corresponding README.MD.
  4. Enter the scale_out folder, and then run the scripts to generate insights of multicore scale-out and compare accuracies of using different ML algorithms, following the corresponding README.MD.
  5. Enter the ilp_solving folder, and then run the scripts to generate insights for NF state placement, following the corresponding README.MD.
  6. Enter the variable_reorder folder, and then run the scripts to generate insights for memory access coalescing, following the corresponding README.MD.
  7. Enter the nf_colocation folder, and then run the scripts to generate insights for NF colocation, following the corresponding README.MD.

How The Artifact Supports The Paper

Artifact Available

Author-created artifacts relevant to this paper have been placed on a publicly accessible archival repository.

The artifact is publicly available via the GitHub link provided above.

Artifact Functional

The artifacts associated with the research are found to be documented, consistent, complete, exercisable, and include appropriate evidence of verification and validation.

The authors included all the key modules of Clara to predict computation performance of NFs in SmartNICs and suggest offloading strategies to further improve its performance. The reviewers all enabled to run them.