Inside a Synthetic Simulation Platform

Recently, we introduced a synthetic simulation-only Simics target machine called QSP (Quick-Start Platform) and it's included in the latest version of Wind River Simics. The idea of QSP is to give every user a useful Simics target that allows them to immediately start using Simics and begin reaping the benefits of simulation for software development. In this blog post, we'll be taking a look under the hood of QSP to see how it works.

The idea behind QSP is to design a piece of virtual-only hardware that is as simple as possible, while still running real operating systems similar to how they work on ordinary hardware.

This is not a new idea. The Simics team and other simulation providers have done similar things in the past - but never with the bottom-up design done with Simics QSP. Usually, designs have started with a system controller and core complex from an existing platform, and then using synthetic devices for various IO. Looking back at my own work history, I ran a student project back in 2006 to port SMP Linux onto a cut-down Simics PowerPC platform. More recently, some colleagues of mine have ported VxWorks to various simple uniprocessor Simics models that basically just contained a processor, memory, serial port, and interrupt controller (modeled closely to existing PICs). The VxWorks team has used a highly-simplified platform in their OS porting to new architectures.

What is truly new with Simics QSP is the scope and depth of the design effort. Instead of cutting down an existing hardware model to something less complex, we started from a blank page and designed each device in the system to be as simple and easy to use as possible. We also wanted the devices to be flexible and scalable and not impose unnecessary accidental limitations on the system setup. After providing the specifications to engineering, I expected to get back a platform with at most a handful of simple devices. However, it turned out that a modern OS requires a surprising number of hardware services in order to work, and the resulting initial QSP design contains nine types of devices in addition to the processor cores.

The picture above shows a block diagram of the QSP that we are now shipping. We can see that it contains quite a few pieces of hardware. To make an OS run, we need a processor. A processor needs RAM to store code and data. To get periodic interrupts, it needs a timer (even though the basic needs are taken care of by internal features within the Power Architecture, timers are needed for ARM and possibly other architectures). To handle interupts from devices and inter-processor interrupts in a multi-core design, you need an interrupt controller. To start non-primary processors in a multi-core system, you need a system controller. A real-time clock (RTC) is needed for sane dates, especially important when using disks to store a large file system. 

Apart from this core set of devices, serial, Ethernet, and LEDs are needed to provide input and output facilities. Disks and flash disks were added to support large file systems and flash-based applications. Not all of the I/O units will be present in all configurations, as that is up to the user to decide.

For added flexibility, we made sure that the number of processors and important functional devices can be varied in case some application needs several of these elements.  It is a virtual platform, and not subject to the physical limitations of real hardware. We wanted to make sure this benefit was passed on to our users. This is where writing both the BSP and the target hardware was most beneficial. Most real multicore hardware designs have arbitrary limitations in system controller registers or interrupt controller designs that limit how many cores can be used, even when a simulator should allow you to use any number of cores. By designing a very scalable hardware interface and the BSP to drive it, we can scale the system to theoretically any number of cores. The current hard stop is at 128, but this might change in the future.

If we look at the design of the devices themselves, it turned out that we needed more details than what I expected at the start of the project.

For instance, Ethernet requires both receive and transmit interrupts to be present.  In an ideal world, transmit would seem unncessary, right?  But we have to model finite bandwidth, as infinite bandwidth is very confusing to existing software, and that means that packets can take time to transmit, and thus transmit-completed interrupts are interrupts (or equivalently, a transmit complete flag along with a polling driver).  We also have descriptor tables in RAM rather than a simple buffer inside the device, as that is what the existing driver stacks for networking would expect. 

Another surprise to me was the fact that simulating (NOR) flash memory is best done by actually implementing a flash command set.  File systems and flash driver stacks expect a flash to follow a certain pattern of operation, such as requiring several writes to turn into write mode.  Just modeling flash as persistent RAM memory does not work, even though it seems the obvious thing to do.

Overall, as we worked on QSP, we discovered just how much the structure of operating systems is shaped by the design of contemporary hardware interfaces.  The BSP or HAL (hardware abstraction layer) in an OS like Linux or VxWorks does not really allow any hardware to run underneath the OS - it expects certain behavioral patterns that are not necessarily what one might think of as the simplest way to express a functionality in hardware. Indeed, if we had made QSP devices more lightweight, the BSP side of the QSP would have been much more complex and would have required much deeper changes to the OS kernels and device driver structures.

Ultimately, what we've released is a solid piece of virtual hardware that allows application development and other system development tasks to progress without the need to model or use any particular hardware model. Obviously, many tasks that Simics is used for does require a model of a specific hardware board, and in those cases QSP is intended as a companion, not a replacement, for traditional Simics models.

For additional information from Wind River, visit us on Facebook.