Jakob Engblom

Analyzing Manycore Scaling with Simics

In my previous blog post on multicore scaling investigations with Simics, I tested a simple parallel program on a variety of machines. The scaling obtained was not particularly impressive, especially not on a 60-core target machine. In this post, we will use the Simics timeline view to look a bit closer at what is going on inside the target machines. In particular, with respect to operating system scheduling of the target threads. 

Testing Manycore Scaling with Simics

A few years ago, I did a Simics demo where I tested the scalability of a multithreaded program as the target hardware went from two to four to eight cores. Unfortunately, I could not take it beyond that point, since the hardware platform that I used simply did not allow for more than eight cores. Now, with the Simics Quick Start Platforms (QSP), the situation is different. I picked up the demo again, and pushed it to sixty (60) cores with ease.

IMA on Simics: An Interview with Tennessee Carmel-Veilleux

After Summer, I have been in contact with Tennessee Carmel-Veilleux at the École de technologie supérieure in Quebec, who has been doing some really cool work with Simics.  Tennessee has learnt Simics very quickly, and has been very clever in how to apply Simics to the research problems he has encountered. In this interview, he tells us some more about his research and use of Simics.

Variable and Deterministic

A virtual platform like Wind River Simics is generally designed to be deterministic. Determinism brings a host of benefits to users (repeatability of bugs, reverse execution, bug transportation, etc.), but is also easily misunderstood. Quite often, users fear that they will lose an important "feature" of a physical machine - its built-in variation across runs of the same software. If the virtual platform is deterministic, surely it will not expose all the different ways a program can run that you get on a random physical machine? Yes it can.

There is a huge difference between the repeatable determinism of a virtual machine and completely invariant, predictable, predetermined behavior.

Transporting Bugs with Checkpoints

I have a paper about "Transporting Bugs with Checkpoints" to be presented at the S4D (System, Software, SoC and Silicon Debug) conference in Southampton, UK, on September 15 and 16, 2010. The core concept presented is to leverage Wind River Simics checkpointing to capture and move a bug from the bug reporter to the responsible developer. It is a fairly simple idea, but getting it to work efficiently does require that some things are done right.

Interview with Girish Venkatasubramanian

After my blog post on Academic Simics earlier this Summer, I got a very nice reply from Girish Venkatasubramanian of UFL. Turned out that he and his group was doing some really interesting and exciting stuff with Simics, researching into Hypervisor architectures and hardware support. Having been a PhD student myself, I can certainly appreciate the excitement and fun of working in that field. We ended up doing a virtual interview, which I am happy to present here.

Travelling into 64-bit Land with Simics

The new Freescale QorIQ P5020 SoC that was announced this week at the Freescale Technology Forum means that yet another chip family has now moved to 64 bits from 32 bits. This is a familiar scenario that has been played out many times before, starting in the mid-1990s as Sun, IBM and MIPS upgraded their server processor architectures to 64 bits. Before that, we had Intel extending the x86 family from 16 bits to 32 bits and IBM extending the System/360 architecture from 24 bits to 31 bits. Each time, the changed caused software pain for a while as the software is updated to work in the new world with more bits to use.  It is also something that Simics has helped with in the past.

True Concurrency is Truly Different (Again)

A recent article at Ars Technica describes yet another security flaw in Windows. Nothing much new in that respect, but this is indeed an interesting attack in that it is enabled by using multicore hardware. It is not practical on a single processor, demonstrating once again how multicore is fundamentally different from multitasking on a single processor.

Analyzer Analyzed

The big news in Simics 4.4 is the new Simics Analyzer product. Analyzer contains a few different technologies, but the most immediately visible is the new execution timeline view. The timeline view shows the execution history of the software on the target system, making it easy to see what has run where and when. In this blog post, I will try to peek a little behind the scenes to show what makes Analyzer and the timeline view work. 

What is Simics, Really?

As you might have seen, Wind River recently acquired Simics, a product formerly sold by Virtutech. My colleagues Michel Genard and Bill Graham recently blogged on the topic.

Simics can have a huge impact on the product development processes, time-to-market and quality. Apart from the cool things that Simics does to improve the development process, it is also a very interesting technology in and by itself. Since I am a technical marketing manager with a license to compile, I would like to spend some time to explain what Simics is under the hood. In my opinion, knowing how tools work is very useful to the engineer considering their adoption or tasked with using them.

Simics is a simulator - a piece of software simulating a piece of hardware. A Simics solution is often called a virtual platform, as it provides a virtual hardware platform for running software. One should note that Simics virtualizes embedded hardware in a different way than hypervisors like the Wind River Hypervisor.

A hypervisor expects operating systems to target a particular virtual machine architecture, while a Simics virtual platform is a software simulation of some concrete piece of hardware. Simics is a tool for developing software without needing hardware, and a hypervisor is a way to manage hardware at runtime. Indeed, you can run a hypervisor and its guest operating systems on top of Simics. The Wind River Hypervisor was actually developed using Simics.

The hardware that Simics has been used to simulate range from basic embedded boards with a single processor or SoC, all the way up to servers and rack-based fault-tolerant clusters. The virtual platform is sufficiently complete and correct to fool the target software into believing it is running on physical hardware, and it is fast enough to be useful for regular software development tasks. When I say "target," I mean the hardware that is being simulated.

From the perspective of the target software, Simics looks like hardware. A Simics setup will load the same binaries as would be used on a physical target system, and execute them in the same way they would execute on the physical target.  The software stack includes everything from the initial boot code to hypervisors, operating systems and user-level application code. There is no need to modify the target code to run on Simics (even though there are intriguing additional possibilities for smart workflows if you are willing to modify the code some).

Simics is a pure software program. It does not require any special hardware, FPGA boards, or "emulators" to function. Simics runs on any PC, anywhere, at any time (at least as long as the PC in question runs Windows or Linux). You can ship a Simics setup literally in an e-mail across the globe. In this way, it can replace hardware boards for global development teams in a fairly magical way.

Inside a Simics Simulation

So what does it actually contain and do?  One large part of a running Simics simulation is the simulation of the target hardware. This contains models of processor cores, buses and other interconnects, memories, peripheral devices, and networks.

A core component of Simics is its fast instruction-set simulators (ISS) which are used to execute the ARM, DSP, MIPS, Power Architecture, SPARC, x86/IA, or other processors' binary code. However, just an ISS does not let you run an operating system. Simics setups therefore also include simulation models for memory-management units (MMU), as well as all the memories and devices found in the memory maps of the processors.

Let's look at a common case, a processor doing a memory operation. When a processor issues a memory load or store, the address to access is first translated through the MMU, resulting in a physical address. This physical address is used to construct a memory transaction in Simics.

The memory transaction is sent to a memory map, which determines what part of the target system the transaction is targeting. If the transaction hits memory, the contents of the memory is read or modified.

If the transaction hits a peripheral device, the simulation model for that device is called to determine the effects of the access. The device model will work out the effects of the access. It might start a timer, send an interrupt to a different processor in the system, reconfigure the hardware, reset a board, send a packet onto a network, drive an IO pin high, or any other effect on the target system.  If the transaction does not hit anything, the simulated process could get an exception back to report the bus error to the software.

Looking at how Simics works as a software program on the host PC, all of this happens as a chain of function calls between the objects making up the simulation model.   Everything in a Simics simulation is an object, using function calls to communicate both transactions related to the target system (like memory operations) and for internal, maintenance, logging, tracing, and other operations. In the hardware design community, this style of simulator modeling is known as TLM, or transaction-level modeling. The essence of TLM is to use move data around in units of transactions, not of clocks or pins or other low-level hardware concepts.

Unlike any other virtual platform solution, Simics can add new objects to a simulation at any point in time. It is also possible to reconfigure how the objects are connected, and modify their properties at run-time.

This dynamic configuration makes it possible to simulate anything that can happen in the physical hardware system, including operations like adding and removing boards from a rack or changing how network cables connect boards together.

To support dynamic change, all Simics objects are created from classes loaded at run-time. Every Simics model is stored in its own .dll or .so file, and can be loaded at any time. A user does not need to recompile anything to create a new configuration, just load the required modules during a Simics run. Simics is actually fairly similar to the Java and .net environments, in terms of how objects are compiled, loaded, managed and connected.  It is not like the static linking model employed in simple C and C++ programs.

One particular advantage of the Simics approach is that each simulation module can be compiled separately, making it very cheap to recompile after a change to the simulated target system.

Modules for Simics can be created in C, C++, Python, or the Simics device modeling language DML. DML is really a C-code generator that automates the creation of much of the boilerplate code needed to build a device model. Other languages can also be used, as long as they can link to C-language modules.

To manage the complexity of the modeled hardware, Simics allows a virtual platform creator to use a special kind of object known as a component to create logical groupings of hardware units. Components group devices, memories, interconnects, and processor cores together into logical units corresponding to chips, SoCs, ASICs, boards, mezzanine cards, racks, and other hardware units. Components can be reused and arbitrarily nested, modeling any form of hardware design hierarchy. By traversing the component hierarchy, it is easy to understand the structure of the (virtual) hardware system.

What can be modeled?

Essentially, Simics can model any system. Multiple processors, boards, networks, and heterogeneous computer architectures and target operating systems pose no problem. The largest setups used by our customers have included up to a thousand processors. The longest simulations have covered many months of target time. The most heterogeneous simulations include tens of different types of processors from several different families.  Multi-core, single-core, symmetric shared memory, asymmetric multiprocessing, 8-bit or 64-bit, Simics have been used to simulate it all.

Adding models of new targets can be done by Wind River, a Simics user, or third-party consultants.  There is no hidden magic; any Simics user can extend the library of hardware models available in Simics with whatever components they need. You do need to buy the Model Builder product in addition to the basic Simics product to build new target models.

Input and Output

To provide input to the simulated system, Simics provides interactive sessions to the target serial consoles and graphical displays. It is also possible to connect simulated serial connections and Ethernet networks to physical networks, to get the real world involved.  The most common model is to keep target I/O automated and reproducible using scripts and various forms of traffic generators. If you count changes to the target software as input, the loading of newly compiled target system binaries to simulated memories and disks is a major form of input which Simics performs directly, without involving the target software. This paragraph is not exhaustive, there are many other (usually domain-specific) ways to produce stimuli for Simics targets.

Simics also has a user-interface side comprising a scriptable command-line, Python-language scripting, a simple GUI, an Eclipse GUI, and connections to debuggers like Wind River Workbench over protocols such as WDB and gdb-serial. These interfaces let users manipulate the target system and debug software running on it, without the target software being disturbed.

Extensibility

Everything in Simics is an object, including the user-interface components, debugger connections, and even the command-line interface itself. All Simics objects are able to use the Simics API and function calls into other objects to do pretty much anything.  This generality makes Simics very flexible and very easy to extend to support new workflows and new customer use cases.

Any Simics user (with the appropriate Simics products) can extend the functionality of the simulator with custom extensions such as specialized trace modules, fault-injection frameworks, connections to their favorite obscure debugger, remote control frameworks, or even custom graphical user interfaces.

Further Reading

This blog post, long as it is, can only scratch the surface of what Simics can do and how it works.  With this orientation, you should be able to go on and read the white papers available on particular aspects of Simics.

There is a also an ESC 2008 class I taught on simulation of embedded systems which provides a deeper conceptual introduction to the matter of simulation.