Mark Hermeling

Migrating to Multi-core using Embedded Virtualization

So, let's assume that you are currently on a single-core board and you would like to explore the possibilities of delivering a next generation device using multi-core technologies. There are a lot of different ways to approach this. The most elaborate involve doing an architecture study into ways in which you could evolve your current device architecture. This study would need to be approached from both a technical, as well as a business level.

Embedded Virtualization in Gaming

A few weeks ago I found myself in the Hard Rock Hotel in Las Vegas. My reason for being there was a seminar that Wind River was organizing together with Intel and AVNET specifically focussed on companies that build gaming. The main topics were graphics, manageability and of course, virtualization.

Consolidate, Consolidate, Consolidate

Many telecom applications are actually built up from multiple smaller sub-applications, often running on their own server in a rack, ATCA or otherwise. These servers run on multi-core processors, depending on the age of the last refresh this could be a dual, quad core or more. This is of course nothing new, what's new is how virtualization can improve server utilization.

Bill-Of-Material

Bill-of-material is something that is important in many devices and I have argued before that virtualization can help with this. Say you have a medical device like an MRI scanner, it is not uncommon that this device has three processors that collaborate. One processor, often in a separate box like an industrial PC, runs MS Windows and is the operator interface, possibly with touch-screen GUI. This is where the results are displayed.

Leaders and Followers

Events such as ESC in San Jose this week are a great way to talk to a lot of customers in a very short timespan. Hence, a great place to be for a product manager like myself. The conversations show a clear difference between leaders in multicore adoption and followers of that adoption.

So, what does _your_ software architecture look like?

Customers often ask me in my opinion as to what their path to multi-core should be. Invariably I ask them two things. 1) Describe your current hardware architecture, your next hardware architecture and what your hardware architecture will look like in 3 years; 2) Describe your current software architecture and any plans you have to evolve it.

This leads to interesting discussions, most customers can draw their hardware architectures, some can white board their software architectures easily, some have more problems, but I have a strong feeling that their drawing differs significantly from the actual implementation.

ESC Silicon Valley

Looking forward to my trip to ESC Sillicon Valley next week. It is promising to be a busy show, especially since ESC is now combined with the Multicore expo. I just leafed through the agenda (in the form of a Nxtbook) and found a large number of sessions that I want to attend, experience show though that I'll probably be too busy talking to customer to attend sessions, which is a good problem to have of course.

I am hosting a 4 hour session (with several of my colleagues) on Multicore Demystified on Tuesday afternoon 2.30-7pm (there will be refresments!) in the Hilton Plaza Room. Do stop by either the session, or our booth at Multicore Expo for a chat if you want to brain storm about your next generation devices.

Virtualization and Fault Handling

In several of my previous posts I have written about the fact that embedded virtualization has low overhead, maintains determinism and all that good stuff. I have also written about some of the benefits of virtualization due to partitioning, scalability and such.

However, there is one aspect of virtualization that gets little 'air time' and that is the fact that there the hypervisor really is a nice, lean, partitioned management layer of your multicore (or single core) hardware.

A sea of cores, now what?

A great number of cores gives more processing power, but this power needs to be harnessed. The easiest way to control a sea of cores is of course to run a single operating system over this sea in a Symmetric MultiProcessing mode (SMP). Most modern operating systems support this (which includes Wind River Linux and VxWorks of course).

The trick is to minimize the impact of Amdahl's law. This law describes the maximum theoretically attainable speedup on your multicore processor. Given a N core multicore processor, one would expect an application to run N times as fast, at least ideally.

Article: Multi-Core Slow Down

An interesting article by Dan Woods on Multi-core slowdown. The article tries to temper people's expectations with regards to multicore. The basic argument goes: A multicore processor has more raw processing power, but it requires the software load that runs on top of that processor to be able to use those cores, if not, the software could run at the same speed as single-core, or even slower.

One of the ways to use all the cores of course is multi-threaded programming in combination with an SMP operating system that can schedule over all the cores (SMP being Symmetric Multi Processing). Typically multi-threaded programs use multiple threads of execution and use synchronization primitives to make sure executions happens in the right order. The last thing you want is 2 threads 'simultaneously' modifying the same data structure. The thing is, one a single core processor, there is no true parallelism, the operating is pretending to do multiple things at once. When you introduce multicore processors you will have true parallelism, this may lead to problems.

One problem occurs when the application uses a lot of synchronization and the result is that the cores are continuously waiting on each other. You are now executing a largely serial application on a multicore part and Amdahl's law will dictate that you will have very little speed-up from the use of multiple cores.

Another possible problem is that your application does not use sufficient synchronization and the result is that the application will do things truly in parallel that were never meant to be executed in parallel. This often leads to data corruption and hard-to-find faults. Not a desirable situation either.

The statements in the article have more of an IT focus, however, they hold for embedded as well. Multicore could lead to a slow-down if the application is unable to use all this new processing power. However, in most cases there is a way to use the multiple cores and to make a difference, either in performance, or in the set of capabilities of a device.

One of the important points to consider when looking at multicore in embedded are the multicore software configuration. The most simple of these multicore configurations uses SMP which allows a single application to use multiple cores, if that application is written for it, as mentioned before. This seems to be the main focus of the original article.

However, other configurations allow customers to run multiple different operating systems on top of these multicore processors. There operating systems could be of the same type (say, VxWorks), or other types (say VxWorks and Linux). Each operating system runs in isolation and will use some of the raw processing power of that processor. Every operating system can of course run one or more applications.

Now, you may ask, why do I need to run multiple operating systems, can I not run multiple applications in a single SMP operating system? Yes you can, however, there are many situations where a single SMP OS is not the best solution because of safety, security, separation, robustness, performance or scalability reasons. Running multiple operating systems gives you, the user, better control as to how to use the power of your multicore processor.

One straight forward example of the use of multiple operating systems is that it allows a customer to use some of the cores in a multicore processor to add more functionality to an existing product using a different operating systems. For example, graphics capabilities using Linux added to the real-time control for an industrial application, or networking to a consumer device. This capability can provide benefits in many different industries, for example consumer applications, network switches, industrial applications, aero-space and defense, the examples are many.

Running more operating systems provides more flexibility such that you can provide more features or more performance to your customers and through that deliver a more competitive, higher value product.

Breadth and flexibility here are important. There is no silver bullet. SMP, uAMP, sAMP, virtualization, all are configurations in the tool-box of the embedded developer and can be used in many different combinations and situations.

There are a a couple of challenges that become immediately apparent:

• The first challenge is to use the right configuration at the right time (a screw driver does not work well if you need a hammer)
• The second challenge is to make sure that everything is integrated, the operating systems have to be optimized for the virtualization layer as well as the silicon
• The last, but certainly not the least is that with these additional configurations, we still need tools for development, analyze, debug and test, that need has not gone away

To summarize: a slowdown when using multicore is certainly possible, but there are ways to solve that slowdown:

1. Make your make your application aware of the multiple cores
2. Using the multiple cores to do new things, possible with new operating system