LITMUS^RT: Linux Testbed for Multiprocessor Scheduling in Real-Time Systems

Dr. James H. Anderson & Students, The University of North Carolina at Chapel Hill

NOTE: This web page discusses an older version of LITMUSRT. Please use the current version unless you have specific interest in the 2008 series.

About

The LITMUSRT project is a soft real-time extension of the Linux kernel with focus on multiprocessor real-time scheduling and synchronization. The Linux kernel is modified to support the sporadic task model and modular scheduler plugins. Both partitioned and global scheduling is supported.

Goals

The primary purpose of the LITMUSRT project is to provide a useful experimental platform for applied real-time systems research. In that regard, LITMUSRT provides abstractions and interfaces within the kernel that simplify the prototyping of multiprocessor real-time scheduling and synchronization algorithms (compared to modifying a "vanilla" Linux kernel). As a secondary goal, LITMUSRT serves as a proof of concept, showing that algorithms such as PFAIR can be implemented on current hardware. Finally, we hope that parts of LITMUSRT and the "lessons learned" may find value as blueprints/sources of inspiration for other (both commercial and open source) implementation efforts.

Non-Goals

LITMUSRT is not a production-quality system, and we have currently no plans to turn it into one. LITMUSRT is not "stable," i.e., interfaces and implementations may change without warning between releases. POSIX-compliance is not a goal; the LITMUSRT-API offers alternate system call interfaces. While we aim to follow Linux-coding guidelines, LITMUSRT is not targeted at being merged into mainline Linux. Rather, we hope that some of the ideas protoyped in LITMUSRT may eventually find adoption in Linux.

Current Version

The current version of LITMUSRT is 2008.3 and is based on Linux 2.6.24. It was released on 09/24/2009 and includes plugins for the following scheduling policies:

Please refer to the download and installation sections for details.

Earlier versions (2007.1 — 2007.3), which are based on Linux 2.6.20 and support additional scheduling policies, are discussed on a separate page dedicated to the LITMUSRT 2007 series.

The first version of LITMUSRT, which was implemented in Spring 2006, is based on Linux 2.6.9.

Development Plans

Re-basing to the then-current Linux kernel version is scheduled for Spring 2010. There are plans to port LITMUSRT to PowerPC and ARM platforms. Please contact us for details.

Support

The LITMUSRT development effort is being supported by grants from, SUN Corp., Intel Corp., IBM Corp., The National Science Foundation (grant CCR 0615197), and The U.S. Army Research Office (grant W911NF-06-1-0425).

Collaborators

The LITMUSRT project is led by Dr. James H. Anderson.

The implementation effort is carried out by students of the Real-Time Systems Group at the University of North Carolina at Chapel Hill:

(Additional collaborators contributed to earlier versions of LITMUSRT.)

Publications

  1. B. Brandenburg and J. Anderson, “On the Implementation of Global Real-Time Schedulers”, Proceedings of the 30th IEEE Real-Time Systems Symposium, pp. 214-224, December 2009. Postscript. PDF. Longer version with all graphs: Postscript. PDF.

    For reference, all evaluated plugins are provided as part of the following patch (against version 2008.3).

  2. B. Brandenburg and J. Anderson “Reader-Writer Synchronization for Shared-Memory Multiprocessor Real-Time Systems”, Proceedings of the 21st Euromicro Conference on Real-Time Systems, pp. 184-193, July 2009. Postscript. PDF. Long version with blocking terms: Postscript. PDF.

  3. J. Calandrino and J. Anderson “On the Design and Implementation of a Cache-Aware Multicore Real-Time Scheduler”, Proceedings of the 21st Euromicro Conference on Real-Time Systems, pp. 194-204, July 2009. Postscript. PDF.

  4. M. Mollison, B. Brandenburg, and J. Anderson “Towards Unit Testing Real-Time Schedulers in LITMUSRT”, Proceedings of the Fifth International Workshop on Operating Systems Platforms for Embedded Real-Time Applications, pp. 33-39, July 2009. Postscript. PDF.

  5. B. Brandenburg and J. Anderson, “A Comparison of the M-PCP, D-PCP, and FMLP on LITMUSRT”, Proceedings of the 12th International Conference on Principles of Distributed Systems, pp. 105-124, December 2008. Postscript. PDF.

  6. B. Brandenburg, J. Calandrino, and J. Anderson, “On the Scalability of Real-Time Scheduling Algorithms on Multicore Platforms: A Case Study”, Proceedings of the 29th IEEE Real-Time Systems Symposium, pp. 157-169, December 2008. Postscript. PDF.

  7. B. Brandenburg and J. Anderson, “An Implementation of the PCP, SRP, D-PCP, M-PCP, and FMLP Real-Time Synchronization Protocols in LITMUSRT”, Proceedings of the 14th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications, pp. 185-194, August 2008. Postscript. PDF.

    Note: The work described in this paper took part in a branch that is currently not part of the main distribution. For reference, we provide the branch as a separate download:

    Please don't use this version for active development. If you are interested in this work, it would be best to first port the desired features to a current version of LTIMUSRT and merge them into the main distribution.

  8. A. Block, B. Brandenburg, J. Anderson, and S. Quint, “An Adaptive Framework for Multiprocessor Real-Time Systems”, Proceedings of the 20th Euromicro Conference on Real-Time Systems, pp. 23-33, July 2008. Postscript. PDF.

  9. B. Brandenburg, J. Calandrino, A. Block, H. Leontyev, and J. Anderson, “Real-Time Synchronization on Multiprocessors: To Block or Not to Block, to Suspend or Spin?”, Proceedings of the 14th IEEE Real-Time and Embedded Technology and Applications Symposium, pp. 342-353, April 2008. Postscript. PDF.

    Extended version, including all graphs: Postscript, PDF.

  10. B. Brandenburg, A. Block, J. Calandrino, U. Devi, H. Leontyev, and J. Anderson, "LITMUSRT: A Status Report", Proceedings of the 9th Real-Time Linux Workshop, pp. 107-123, November 2007. Postscript. PDF.

  11. B. Brandenburg and J. Anderson, "Integrating Hard/Soft Real-Time Tasks and Best-Effort Jobs on Multiprocessors", Proceedings of the 19th Euromicro Conference on Real-Time Systems, pp. 61-70, July 2007. Postscript, PDF.

  12. J. Calandrino, H. Leontyev, A. Block, U. Devi, and J. Anderson, "LITMUSRT: A Testbed for Empirically Comparing Real-Time Multiprocessor Schedulers ", Proceedings of the 27th IEEE Real-Time Systems Symposium, pp. 111-123, December 2006. Postscript, PDF.

Download

The source code of LITMUSRT is made available as open source under the terms of the GNU General Public License (GPL).

The current release of LITMUSRT is 2008.3. It consists of our Linux kernel modifications in the form of a patch against Linux 2.6.24 and liblitmus, the user-space API for real-time tasks, as well as ft_tools, a collection of tools used for tracing with Feather-Trace (which is part of the LITMUSRT patch).

LITMUSRT 2008.3

Based on Linux 2.6.24. Released in September 2009.

Files:

Major changes (since LITMUSRT 2008.2):

Please consult the Change Log for further details.

LITMUSRT 2008.2

Based on Linux 2.6.24. Released in December 2008.

Files:

Major changes (since LITMUSRT 2008.1):

LITMUSRT 2008.1

Based on Linux 2.6.24. Released in July 2008.

Files:

Major changes (since LITMUSRT 2007.3):

Please note that the current implementation is a prototype with certain limitations. Most notably, it is not secure in a multiuser context, i.e., real-time system calls do not require superuser privileges.

Older releases: LITMUSRT 2007 series.

Installation

The current release of LITMUSRT consists of an extension of the Linux kernel that adds support for the sporadic task model, a scheduler plugin infrastructure, and some scheduler plugins, as well as a user-space library that provides the LITMUSRT real-time API. Note that the current implementation only works on the Intel x86-32 and sparc64 architectures.

Patching the Kernel

The extension to the Linux kernel is released as a patch against Linux 2.6.24. To install the LITMUSRT kernel, first download the Linux kernel 2.6.24 and untar it in a directory of your choice (hereafter referred to as $DIR). Second, apply the LITMUSRT patch (see Section Download) and configure, compile, and install the kernel as usual. The patch is -p1 applicable. To summarize, the LITMUSRT kernel can be obtained, patched, and compiled with the following commands:

cd $DIR
# get Linux 2.6.24
wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.24.tar.bz2
tar xjf linux-2.6.24.tar.bz2
wget http://www.cs.unc.edu/~anderson/litmus-rt/download/2008.3/litmus-rt-2008.3.patch
mv linux-2.6.24 litmus2008
# apply the LITMUS RT patch
cd litmus2008
patch -p1 < ../litmus-rt-2008.3.patch
# create a working kernel configuration
#  - select HZ=1000
#  - enable in-kernel preemptions
#  - disable NO_HZ
#  - don't use power management options like frequency scaling
#  - disable support for group scheduling
make menuconfig
# compile the kernel
make bzImage
make modules
# proceed to install kernel, build initrd, etc.
...

When configuring the kernel, note that there is a menu (at the very end of the list) with LITMUSRT-specific configuration options. For reference, we provide a configuration that is known to work under QEMU.

Libraries

The user-space library for real-time tasks, liblitmus, depends on the LITMUSRT kernel kernel and provides its own build system (based on scons). In order to compile liblitmus, you need to adjust the variable LITMUS_KERNEL in the SConstruct file to point to your copy of the kernel.

cd $DIR
wget http://www.cs.unc.edu/~anderson/litmus-rt/download/2008.3/liblitmus-2008.3.tgz
tar xzf liblitmus-2008.3.tgz
cd liblitmus 
# change LITMUS_KERNEL in SConstruct to point to the kernel source
scons

Please refer to the documentation on how to use the LITMUSRT real-time API as provided by liblitmus.

Documentation

Unfortunately, most of the documentation has yet to be written. To get an overview of the architecture of the kernel extension, we recommend reading the paper “LITMUSRT: A Status Report”.

Real-Time Scheduling Policies

The kernel contains the following real-time scheduling policy implementations:

Only one policy can be active at any time. Initially (i.e., during and after boot), the "Linux" policy is active. You can use the tool showsched (part of liblitmus) to display the name of the currently active policy.

Changing the Active Policy

You can use the tool setsched (part of liblitmus) to select a new plugin at run time.

Screen shot of setsched

Only root can change the active policy, and only when there are no real-time tasks present.

If you do not have the dialog utility installed, then you can still used setsched by passing the desired scheduling policy as a commandline parameter, e.g. type setsched PFAIR to activate the PFAIR plugin.

Writing Real-Time Tasks

The user space library that provides the LITMUSRT API, liblitmus, contains two example real-time tasks (base_task.c and base_mt_task.c) that both illustrate how to use the API and provide a skeleton for real-time task development. To get started with development, please take a look these example programs.

Tracing Overheads and Scheduling Decisions

LITMUSRT provides numerous tracing facilities that are discussed in-depth in the tutorial Tracing with LITMUSRT.

Please contact bbb[AT]cs.unc.edu if you have any questions.

Credits

Valid XHTML 1.0 Strict

Linux is a registered trademark of Linus Torvalds.
The LITMUSRT logo was designed by Jasper McChesney of Break for Sense Design.
Web design by Björn Brandenburg.