VxWorks: A Long Love Story with Multi-core

While it may seem mundane today to talk about multi-core, it used to be a big deal. Back in the day, all CPUs came with a single core (Zylog Z80, Motorola 68K, PowerPC 8xx, 386, etc.).  The one thing that CPU manufacturers did to provide more horse power was increase the frequency.  Eventually, the laws of physics made it harder to do, starting the era of multi-core CPUs.

Ever since, our VxWorks real-time operating system has introduced many ways to harness the extra computing power that comes with multi-core CPUs.

The very first thing that Wind River introduced in 2001 was support for unsupervised Asymmetric Multi-Processing generally referred to as AMP or uAMP (see Figure 1). With uAMP, the bootloader brings up independent VxWorks-based applications each running on a single core, side by side. Effectively, one can double the amount the things that can be done on a single board computer (SBC). To enhance the value of the software/hardware combination, Wind River introduced communication mechanisms so that VxWorks instances could “talk” to each other.

Figure 1: AMP configuration

A few years later, in 2007, support for Symmetric Multi-Processing (SMP) was introduced (see Figure 2). SMP shifts the burden of using the cores away from the user to the operating system. Practically, this is done using a load balancer. With VxWorks, the load balancer uses a highest N priority-based scheduler that aims to use all cores most effectively.

Figure 2: SMP configuration

As CPU manufacturers got more creative to provide even more computational power, concepts like hyper-threading were introduced. This means a core has forms of hardware threads – generally 2. So, a dual-core CPU provides access to 4 virtual cores. VxWorks’s load balancer transparently leverages such virtual cores, making it easy for applications to scale.

Figure 3: SMP configuration with CPU threads

In the recent past, a new hybrid multi-processing concept has been introduced in some OSs, sometimes referred to as Bound Multi-Processing (BMP).  While not new in the case of VxWorks, since it’s been there all along since version 6.6, it is an interesting approach.  The idea is to use a single OS, as configured in SMP, but to use affinity to “partition” threads on the cores. This approach can be seen like a form of supervised AMP (sAMP).

Figure 4: BMP configuration

With VxWorks, we use a more classic description for this approach called affinity. Indeed, with VxWorks, each and every thread can be configured to run on a specific core or you can let the scheduler decide.  This assignment is referred to as task CPU affinity.

The default configuration using SMP often provides the best overall load balancing. However, there are cases in which assigning a specific set of tasks to a specific CPU can be useful. For example, if a CPU is dedicated to processing some data and does no other work, the cache remains filled with the code and data required for that activity. This saves potentially precious time used for “moving” a thread to another CPU; which is incurred even within a single piece of silicon, as the L1 cache is bound to a single CPU, and the L1 must be refilled with new text and data if the task migrates to a different CPU.

In order to push the concept further, VxWorks also supports the concept of interrupt CPU affinity. In this case, specific interrupts can be routed to specific cores, enabling applications even more control on how they leverage the hardware.

As we always try to provide maximum flexibility, VxWorks also brings to the table the ability for threads to inherit the attributes of their parent (the thread that created them). This enables, among others, user-space applications, and hence all their threads, to be pinned to cores as well.

Finally, to complete the picture and truly enable a blend of SMP and sAMP, VxWorks also supports CPU reservation. With CPU reservation, the scheduler will never try to use a core reserved by thread. This prevents the dedicated threads from being preempted by other threads in the system, and thus improving performance even more.

Figure 5: SMP configuration and CPU reservation

Our affinity (no pun intended) with multi-core wouldn’t be complete if we did not support true virtualization-backed supervised AMP.  With our Helix Virtualization Platform which leverages our ARINC 653 avionics DO-178C DAL A certified platform, users can harness the power of complete separation on top of all the other possible options VxWorks has to offer.

Figure 6: Virtualization-based configuration

If you have any need for pushing the boundaries of your multi-core system with VxWorks, we have all you need: SMP, BMP, uAMP, sAMP and everything in between.