APLawrence.com -  Resources for Unix and Linux Systems, Bloggers and the self-employed

Realtime-Preempt for the Linux kernel.

© June 2005 drag

Author: drag
Date: Sat Jun 11 08:49:58 2005
Subject: Realtime-Preempt for the Linux kernel.

The Linux kernel has always been somewhat preemptive, but with the 2.6 kernel there were big improvements aimed at making the kernel almost fully preemptive and added the 'preemptive' compiling option, which has lead improvements to user response which is important for a desktop operating system. However these improvements have been less then stellar at reducing system latencies to a low level. Currently the stable versions of Linux 2.6 cannot be relied on to handle scheduling latencies under 100msec. For non-critical apps you can probably get scheduling down under 10-30msec, depending on the actual kernel version. This is better then 2.4 series, but not close to 2.4 series combined with low-latency patches.

Realtime-preempt is a set of patches for the latest version of the Linux kernel (2.6.12-rc6 at this writing). They are being developed by Ingo Molnar (a kernel hacker from Redhat) that brings the pre-emption capabilities to the next level. It's goal is to make the full fledged Linux kernel have as close to real-time response as possible. (As a warning, though, the realtime preempt patches are under going heavy development and have been for some times. It's not unusual for Ingo to update the realtime-preempt patch several times a day, or even within hours of each other. )

The original announcement of this is available here:
(link dead, sorry)
At that time it was called voluntary-preempt, but the goal of the patch has increased significantly since then.

The latest patch is available here:
RT Project

By applying this patch it adds several configurable options to the kernel compile-time config system.

No Forced Preemption (Server)
Voluntary Kernel Preemption (Desktop)
Preemptible Kernel (Low-Latency Desktop)
Complete Preemption (Real-Time)


Thread Softirqs
Thread Hardirqs
Thread RCU
Old-Style Big Kernel Lock

As well as more debugging capabilities to aid in tracking down causes of additional kernel latencies. All the technical details of what all of this does and what all of it is for is a bit over my head.

Some information on various things:
Links to Linux RCU information:
What is RCU, Fundamentally?

A wiki on the subject:

To allow regular users, mostly for audio work, access to realtime-scheduling facilities you can use the Realtime-LSM module. Realtime linux security model takes advantage of the 2.6 series API developed for such things as SELinux to allow users under a certain group access to realtime performance.

Realtime Linux Security Module

The result of all this is that the linux kernel with full-fledged realtime-preempt applied and configured allows scheduled latencies of under 1msec, which is a drastic improvement over the standard kernel. This has dramatic impact for tasks that require a set latency for proper operation. The danger of realtime-lsm and running userlevel programs is that with sufficiently high priorities you can have a buggy program lock up your machine. For isntance I run the jack audio server program (jackd) at a priority of 65, which is higher then my IDE controller, which runs at a priority of 46.

One big is when musicians require set latency when using Linux as a OS in a digital audio workstation (DAW). It's important to have set latencies when your mixing music from a wide variety of inputs and it's very important to have low-latency responses. Anything under 7msec is considered non-noticeable latency by humans and since the software is only part of the latency caused by electronic music (you have the midi controller, the midi line, the sound card, and the synthizer for instance, each adding some latency) it's very important to get the latency as low as possible.

This is why I learned about all this realtime-preempt. I bought a new sound card a while ago, and just recently bought a midi piano-style keyboard controller to go with it. Before applying this patch I had to either set a high latency in my Jack audio server or suffer thru occasional squawks and cracks in the sound. Now with kernel 2.6.12-rc6-git1 with the realtime-preempt patches applied, and tweaks to the hardware (such as setting very high realtime-priority to my sound card's IRQ and increasing the PCI 'latency' (bus bandwidth) to 128) I can get reliable and clean operation down under 2msec of latency. This all functions perfectly well with a 'realtime' cpu load of 50-80 and system load peaking at 100. Depending on the application I have to give certain ones some low realtime scheduling priority of their own to have them keep up with the jackd service though. The only time I get xruns (errors in the audio stream) is when I start or close applications that automatically hook into jackd. The system is a 32bit 2ghz AMD Althon system (2400+) with a gig of ram. With a faster system you should be able to get even better results.

Of course this is not just for audio work, but should be of interest for any person desiring realtime operations. It's a potentially very attractive thing for embedded developers that would want the ability to run realtime operations but still be able retain the full-fledged capabilities (and code base) of a modern PC operating system.

This isn't the only set of patches that seek to get hard realtime capabilities to 2.6. For instance both Monta Vista and Fsmlabs have their own version of realtime linux 2.6, but since Ingor Molnar is a core developer his approach (and his approach is the least drastic without have to add new APIs) would be most likely to be accepted in to vanilla linux by Linus. Although it is very questionable if these capabilities will ever be permanently added to the 2.6 series kernel. The previous low-latency kernel patches from 2.4 series, in comparison, were never meant to be incorporated into Linux proper, but were just a way for audio developers to get the needed performance.

Got something to add? Send me email.

(OLDER)    <- More Stuff -> (NEWER)    (NEWEST)   

Printer Friendly Version

-> Realtime-Preempt for the Linux kernel.


Inexpensive and informative Apple related e-books:

Are Your Bits Flipped?

Take Control of High Sierra

Take Control of OS X Server

Photos for Mac: A Take Control Crash Course

Take Control of Preview

More Articles by © drag

Sun Dec 24 05:35:42 2006: 2779   anonymous

well have this patches ever been translated into FPS for video cards?

Sun Dec 24 14:50:55 2006: 2780   drag

It would probably either have very little effect or have a negative effect on the performance of video games, I figure.

The idea of the 'realtime' style performance is to have a reliable maximum latency to a particular program's operation. That is you want to use this when you care about how long something takes. This doesn't nessicarially translate to making things run faster... The goal is to get it just more reliable.

In order to do this you usually end up sacrificing some of the efficiency of your system and actually get a lower overall performance.

For instance a common usage in Linux would be for realtime audio editing. For example say your making a recording of a person playing a guitar. Then you have recorded vocals and maybe a midi software syth drum machine which the guy is listening to and playing along. So if your doing this all live you can't have everything running out of sync of each other and you can't have the pops and squeals that sometimes happen when your sound card runs out of it's buffer. So latency is very important for doing many types of audio work. So that is a example of a sort of task were these patches are wanted.

In Linux you can use a service called Jackd to act as a way to control and route the audio and midi inputs and outputs to various different programs and hardware. So this is the sort of thing which you would give 'realtime' priority too.

Jack is configurable and you can settup buffers and whatnot to adjust latency.

So the lower latency you allow it the crappier your performance of the machine gets. If you set jack for 5ms (which is probably overkill.) on a fairly modern machine you can eat up all the cpu resources and such even though jack is a very light resource thing normally. If you set it to 30-100ms then you can hardly notice the difference.

Now for a video game this is were you want your machine to be as efficient as possible. There is going to require a certain level of multitasking that you need and that requires preemption and low latency.. For instance your sound system and network latency can be important... But since when your playing the game that is usually the only thing your doing (or maybe in addition playing some music in the background and such) then the normal level of preemptiveness available will probably provide the best performance you can get.


Printer Friendly Version

Have you tried Searching this site?

This is a Unix/Linux resource website. It contains technical articles about Unix, Linux and general computing related subjects, opinion, news, help files, how-to's, tutorials and more.

Contact us

Printer Friendly Version

A man can be destroyed but not defeated. (Ernest Hemingway)

Linux posts

Troubleshooting posts

This post tagged:



Unix/Linux Consultants

Skills Tests

Unix/Linux Book Reviews

My Unix/Linux Troubleshooting Book

This site runs on Linode