There are the steps required to reproduce all the experiments in our paper:

J. Bakita, S. Ahmed, S. H. Osborne, S. Tang, J. Chen, F. D. Smith, J. H. Anderson, "Simultaneous Multithreading in Real-Time Mixed-Criticality Systems", Proceedings of the 27th IEEE Real-Time and Embedded Technology and Applications Symposium, May 2021, to appear. (PDF)

Each step relies on completion of all the previous ones:

  1. SMT Interference Benchmarks for Tbl. 1 and Tbl. 2
  2. Cache Sensitivity Benchmarks for Fig. 7 and Tbl. 2
  3. Schedulability Study for Sec. 6
  4. Taskset Generation for Sec. 7
  5. Case Study for Sec. 7

For full reproduction, each of these steps relies on all of the previous ones to have completed. If any steps are not feasible due to errors or hardware limitations, please double-check that you have the latest code of our code and instructions before skipping ahead. Where possible, we prepopulate each step's configuration so that you can still continue with evaluation even if you are unable to directly derive every configuration parameter.

These steps were last updated .

SMT Interference Benchmarks

These benchmarks measure the slowdown incurred by each task when paired onto the SMT threads of a core. We measure this slowdown under three different system configurations:

  1. With no cache isolation (notationally xi)
  2. With only L3 isolation (via Intel CAT/AMD QoS Extensions) (notationally i3)
  3. With L2 and L3 isolation (via cache coloring) (notationally i)

These benchmarks reproduce Tbl. 1 and the "SMT Effect" entries of Tbl. 2, plus PETi, PETi* and PETi : j for all i and all j. We measure all PETi and PETi : j under all system configurations (xi, i3, and i), but only measure PETi : j values without isolation (xi) as keeping with our system design.

Platform Compatibility An AMD Ryzen 9 3950X-based system is needed for exact reproduction, but any AMD Zen-2-based processor should yield similar results. Cache coloring requires the processor's cache mapping function, and is thus not available on other systems. However, the xi and i3 configurations do not require cache coloring and can be run on any Linux kernel/processor configuration which supports Intel CAT/AMD QoS Extensions via resctrl. Our scripts automatically check for the MC^2 kernel however, and these checks will have to be commented out to run on other systems.

Running the Experiments

  1. Complete Kernel Setup and Benchmark Setup
  2. Adjust your kernel cmdline parameters to isolate the benchmarking cores from all others:
  3. Stop the irqbalance service: sudo service irqbalance stop
  4. Build the benchmarks: cd mc2-scripts-and-benchmarks; make benchmarks
  5. Build and install the cache management module: cd wbinvd; make; sudo insmod wbinvd.ko
  6. Build the contending task: cd thrasher; make
  7. Determine how many execution time samples you want. All our experiments use 1k, but this takes days to run, so we suggest using 100 instead.
  8. Choose a unique identifier for your run, eg ae.
  9. Run all the benchmarks. For an experiment on core 16 (Linux CPU IDs of 15 and 31) with 100 samples and a run identifier of ae:

    sudo ./ 100 ae 15 31
    sudo ./ 100 ae 15 31
    sudo ./ 100 ae 15 31
  10. Wait until the scripts finish. This process is very sensitive to interference, so please refrain from running other work on the system as much as possible. When each script completes, it prints the names and locations of each output file. Record these for later analysis.

Output Format Each output file uses the following format:

Date Hour Isolation type  Run ID
  v   v    v               v
         ^       ^              ^
Contention type  └ Pairing type └ Pair ID (not shown)

Analyzing the Results for Level-A and Level-B We provide scripts in smt_analysis which automatically parse the output files and generate the Mi : j and Mi values for each experiment and isolation type. To get Mi : j values, run with the baseline (PETi) and paired (PETi : j) timing results. For example, to generate the values for DIS from a run on March 9th at 2pm with ID ae, run:

user@machine:/mc2-scripts-and-benchmarks$ ./smt_analysis/ dis/Mar09-16-c-i-ae-A.txt dis/Mar09-16-c-i-ae-B.txt dis/Mar09-14-c-i-ae.txt
Analyzing results using Level-A/B methodology...
Average offset is: 116.90ns
Bench           field      matrix     neighborho pointer    transitive update
field         :      0.397      0.617       0.47    -0.0136     -0.357     0.0846
matrix        :        N/A      0.599      0.789      0.142      0.248      0.341
neighborhood  :        N/A        N/A      0.523        N/A        N/A     -0.293
pointer       :        N/A        N/A        N/A      0.114      0.218     0.0994
transitive    :        N/A        N/A        N/A        N/A     0.0815      0.129
update        :        N/A        N/A        N/A        N/A        N/A      0.136
Overall average is 0.228 with standard deviation 0.287 using `max`
3 of 19 M_i:j values are at most zero - 15.789473684210526 percent
0 of 19 M_i:j values are greater than one - 0.0 percent
For 16 of 19 M_i:j values in (0, 1], average: 0.312 with std. dev. 0.22 (coeff. var. 0.706) using `max`
        bucket       | ________________________________________________________________________________ Total Counts
[0.081493, 0.152198) | ⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀ 7
[0.152198, 0.222903) | ⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 1
[0.222903, 0.293608) | ⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 1
[0.293608, 0.364313) | ⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 1
[0.364313, 0.435018) | ⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 1
[0.435018, 0.505723) | ⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 1
[0.505723, 0.576428) | ⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 1
[0.576428, 0.647132) | ⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡷⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 2
[0.647132, 0.717837) | ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 0
[0.717837, 0.788542) | ⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 1

The first line of results details the average offset between the start of each member of a pair from the other. This should be fairly small. The following is a table of Mi : j values. Next several summary statistics are presented alongside a simple histogram of the Mi : j values. How does this correspond to the values in Tbl. 1?

The Tbl. 1 values also appear in bottom right section of Tbl. 2.

Analyzing the Results for Level-C The process for this is very similar to that used for Level-A and Level-B, except you use and need to specify fewer inputs. For example, to generate the values for DIS from a run on March 9th at 2pm with ID ae, run:

user@machine:/mc2-scripts-and-benchmarks$ ./smt_analysis/ dis/Mar09-14-c-xi-async-ae.txt dis/Mar09-14-c-xi-ae.txt
Reading file using asynchronous pair format...
Bench           M_i
field         : 1.89
matrix        : 2.17
neighborhood  : 2.26
pointer       : 1.51
transitive    : 1.09
update        : 1.54
Overall average is 1.75 with standard deviation 0.408 using `mean`
0 of 6 M_i values are at most one - 0.0 percent
2 of 6 M_i values are greater than two - 33.333333333333336 percent
        bucket       | ________________________________________________________________________________ Total Counts
[1.092161, 1.209424) | ⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 1
[1.209424, 1.326688) | ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 0
[1.326688, 1.443951) | ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 0
[1.443951, 1.561215) | ⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀ 2
[1.561215, 1.678478) | ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 0
[1.678478, 1.795741) | ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 0
[1.795741, 1.913005) | ⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 1
[1.913005, 2.030268) | ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 0
[2.030268, 2.147531) | ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 0
[2.147531, 2.264795) | ⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠀ 2

As Mi values are a representation of the worst slowdown possible against any other task, the table is not all-to-all as with the Mi : j values. The overall average and standard deviation correspond to the numbers in the bottom-right of Tbl. 2.

Cache Sensitivity Benchmarks

These benchmarks are needed to reproduce Fig. 7.

Note: The 3-D plot inset to Fig. 7, the 3-D plots in the supplemental appendix, and the derived "Cache Sens." entries in Tbl. 2 can also be generated via a variant of these benchmarks, but the experiments suffer a combinatorial explosion and are extremely lengthy. Instructions are not included here, but can be made available upon request via the AE PC chair.

Platform Compatibility Exact reproduction requires the AMD Ryzen 9 3950X and the MC^2 kernel, but similar results should be evident on any Linux system with a Intel CAT/AMD QoS Extension-supporting processor. Our scripts automatically check for the MC^2 kernel, and these checks will have to be commented out to run on other kernels.

Running the Experiments

  1. Complete Kernel Setup and Benchmark Setup
  2. Build the benchmarks: cd mc2-scripts-and-benchamrks; make benchmarks.
  3. Build the memory-thrashing contender: cd thrasher; make; cd ...
  4. Run the experiments. For a 100 sample run (paper uses 1k, but even 100 takes several hours) with ID cache-ae on core 16 (CPU ID 15): cd dis; sudo ../ -m dis -p 15 -l 100 -b dis2MbInNames.txt -B -W inputs/4mb_WSS -T in0 -C inputs/caches_all cache-ae (see the help text of for additional parameter detail).

Plotting the Results

  1. The plotting script requires Python 3, matplotlib, and numpy. To install these on Ubuntu: sudo apt install python3 python3-matplotlib python3-numpy.
  2. Reformat the results for plotting using For an experiment run at 2pm on March 9th: ./ Mar9-14-c-xi-cache-ae.
  3. Plot the results: ./ Mar9-14-c-xi-cache-ae-pp.txt.clean
  4. See the output file for a reproduction of Fig. 7

Schedulability Study

These experiments are needed to reproduce the plots in Sec. 6 and our online appendix. If you are unable to run SMT Interference Experiments and Cache Sensitivity Benchmarks, the data from Tbl. 2 in our paper has been prepopulated.

After completing Schedulabilty Framework Setup, see mc2-sched-study/ for detailed instructions on running, plotting, and summarizing the output.

Taskset Generation

These experiments use the schedulability study code to generate tasksets for our case study in Sec. 7.

After completing Schedulabilty Framework Setup, see mc2-sched-study/ for detailed instructions.

Case Study

These experiments test the holistic viability of our system by running and checking for the tardiness of some schedulable task sets from our schedulability study. If you were unable to complete Taskset Generation, the tasksets for our paper are included and can be used instead.

This requires an AMD Zen-2-based processor with at least 8 cores and SMT.

Running the Case Study

  1. Complete Kernel Setup, liblitmus Setup, and Benchmark Setup
  2. Set the path to liblitmus: export LIBLITMUS=$(pwd)/liblitmus
  3. Build the case study by running make case-study from mc2-scripts-and-benchmarks
  4. Switch to the MC^2 scheduler by running sudo setsched MC2 from liblitmus
  5. Start the prepackaged case study by either:
  6. Each task set task 80 minutes to run (10m initialization, 60m run, 10m cleanup). It will take 13 hours and 20 minutes to run all the pre-packaged task sets.
  7. Monitor the output for deadline misses. The log format is (<program name>/<pid>:<job number>) <log message>. If a job is tardy, the log message includes the absolute and relative tardiness. Otherwise, regular liveness messages are printed by the Level-C tasks. (You can review _rt_start_loop() in mc2-scripts-and-benchmarks/extra.h to see how this works.)
  8. If the task sets all run to completion and no Level-A or -B jobs miss their deadlines, the case study has successfully completed!

Note: We have found our scheduler to be highly robust to non-real-time interference, so it should be safe to continue running other work on the system while the case study executes.

Kernel Setup

Obtain, build, and install the kernel sources by running:

git clone --branch rtas21-ae
cd litmus-rt
sudo ./ # This installs kernel build dependencies
make bzImage modules -j8
sudo make INSTALL_MOD_STRIP=1 modules_install install # This copies the kernel to /boot/

To boot with the kernel, simply reboot and select the mc2-v2 kernel from your bootloader.

Note: We do not support graphics drivers when page coloring is enabled, so plan to only interact with your system over SSH, serial, or some other remote access protocol after rebooting.

Compatibility Note: If you are not running on an AMD Zen-2-based system, should print a warning and automatically disable page coloring.

liblitmus Setup

To download and make this library, run the following:

git clone
cd liblitmus

Note: You must complete the Kernel Setup first and have python on your PATH.

Benchmark Setup

Download our code and documentation by running:

git clone  --recurse-submodules --branch rtas21-ae
cd mc2-scripts-and-benchmarks

Schedulability Framework Setup

This framework is needed for the Schedulability Study and Taskset Generation. It requires Python 3.6 or newer, NumPy, Matplotlib and Gurobi with Python bindings. Gurobi is commercial software, however free academic licences are available. Please follow these instructions to obtain Gurobi. Please see your package maintainers on how to obtain a recent version of Python if your current version is too old. If running on Ubuntu, sudo apt install python3-matplotlib python3-numpy is sufficient for the other dependencies. We do not support running on Windows.

After installing the dependencies, download our code by running:

git clone --branch rtas21-ae mc2-sched-study
cd mc2-sched-study