What’s the 4-1-1 on 3.1.1?

We have just announced that we are taking orders for Wind River Workbench On-Chip Debugging version 3.1.1. Here's some infomation (i.e. "the 4-1-1") on this release. Version 3.1.1 is a significant update to the software and firmware that powers our JTAG debugger units – the Wind River ICE 2 and the Wind River Probe. A couple of the highlights include our support for RMI Corporation's XLR and XLS processor lines, a new capability that strengthens our support for multicore and multithreaded processors. This component of the release was the result of some hard work and great teaming between the folks at RMI, our professional services organization, and our engineering team in Canton. The other thing that has our team excited is that version 3.1.1 introduces support for Intel Architecture starting with support for the Intel Atom product family. Intel Architecture support in our on-chip debugging solutions extends our existing coverage for ARM, MIPS, and PowerPC. These different architectures are supported using firmware updates to the same physical hardware – this is something that our clients have told me that they appreciate since the same debug solution can be used across multiple projects in there heterogeneous environments. I tend to feel a certain amount of pride everytime we execute a new release, but I'm particularly proud and nostalgic about this release. I'm proud because our clients are jazzed that we are expanding our optimized-for-multicore JTAG debugging solution to the Intel Architecture and are pleased that we have a non-intrusive debug solution for Intel that helps to abstract the debugging of operating systems and the stuff running on them – including operating systems like VxWorks and Wind River Linux.

So why the feelings of nostalgia? It is because the 80×86 instruction set was the first architecture I worked with.

Coming out of university, I started my professional and embedded software career with a company in Montreal that developed point-to-point TDMA-based wireless networks. The company had been primarily a hardware company, but because this "software thing" seemed important, they were ramping up their software engineering team. When I was brought in, I was asked to study if-and-how C++ could be used in this environment. We were also asked to investigate techniques to improve the quality of our software design processes by using formal requirments management techniques, OO-techniques, and state machines to improve the quality of our design activities.

In those first few weeks of my employment, my mornings would consist of researching new software engineering techniques and my afternoons were spent in the lab learning how to debug the distributed system. At this time, most of the system was running on an Intel 80C186 processor and the software consisted mostly of undocumented assembly language that had been written by a former employee who had left to pursue other interests because he had become independently wealthy. (This was the polite explanation that was given to me)

The afternoons were – let's just say – "difficult". My mentor "V" was a brilliant engineer, but not much of a talker. I would later describe "V" as a man who was put on earth with a limited number of words that he could speak for his entire life, and he was using each and every one of these words sparingly. So my afternoons would be spent watching "V" debug the system from a UNIX terminal that was connected to one or more HP 64700 emulators attached to either a central station or a terminal station. "V" was also a six-sigma UNIX blackbelt and a smoker. So, we would sit together at this terminal – with smoke billowing all around the screen – and I would watch as "V" would type a flurry of shell and emulator commands that would grab code from SCCS, compile it in the background, load it into the emulator, and then display assembly code in the debugger window. I would struggle to keep pace as "V" entered a flurry of commands – vi, grep, sed, make, get, sccsdiff, "yy", "set trigger on read/write 0F0B83E00H", "set trigger on I/O output to 040H"  - while navigating all over the filesystem. It was intense – so intense that it didn't feel appropriate to ask questions while all this activity was going on.

I was totally and completely lost.

Usually after an hour or so of analyzing the 80×86 assembly code and traces, "V" would speak saying something like:

"Ah-ha! Can't you see it?! The blah-blah hasn't received a message from the blah-blah, and the connection memory updates are not getting to the blah-blah. This is why we don't have dial-tone."

There would then be another flurry of typng activity culminating in a "make" command. And then he would walk away. I usually used this time to review the history of the commands "V" had typed in the shell. In hindsight, it really felt like trying to understand the hieroglyphics left behind my another civilization.

This went on for weeks (it seemed longer) and then due to some scheduling screw-up, all of the senior software engineers who knew the system were on vacation at the same time. At that time our engineering director – who looked like a Hellenic version of the Malboro Man – was a real "getting things done" kind of guy. There was an ASIC for our next generation terminal station that had to get done and they needed some software guys to test it out. The director walked into the pit where the new engineers were researching software engineering techniques and I remember the discussion going something like this:

Malboro Man: "Sonny, what are you doing for a living?"

Me: "I'm reading about C++ for our next generation software development efforts…"

Malboro Man: "Sonny, does this look like an [expletive deleted, gerund form] library?! 'M' needs to debug the ASIC in the lab before he can go on vacation. Go help him."

We walked to the lab together and I'll never forget the look on "M's" face when he saw that some new grad was between the debugging of his $100K ASIC and a much needed vacation with his family. The next week and a bit was challenging, but proved to be rewarding. We connected the HP 64700 emulator to the new board.This activity in itself was noteworthy just based on the size and cost of these emulators (see figure below). As a new grad, I always felt nervous about dropping one of these emulators that cost about the same amount as my annual salary at the time. Because they were so big, they were almost always mounted in some dangerous configuration.


With the emulator mounted to the board with the new ASIC, I sat down in front of the UNIX terminal, brought out my notebook of commands that I captured, and together with "M" we somehow (read: miraculously) verified the ASIC and "M" went on vacation with his family. For me the bigger accomplishment was learning from the experience of debugging the system at the HW-SW interface level. I quickly developed an appreciation for the visibility that on-chip debugging provides into the system. This knowledge of the system helped  me in later phases of my career as we designed both the
hardware and the software for our next generation systems. Reading the specs is one thing, but using an on-chip debugger to gain insights into the execution of the software proved to be essential capability required to help verify and optimize our software.

This year's TechInsights Embedded Market Study by TechInsights confirms that "Testing and Debugging" activities continue to take up the most time in our embedded software development products. Embedded developers need quality debugging tools to help address these challenges and meet their schedules. What's also encouraging is that when asked the questions "Which of the following are your favorite/most important software/hardware tools", "debuggers" rated just a hair behind "compilers" and "JTAG/BDM" debuggers finished in the top-five. These on-chip debugging solutions are essential for embedded and device software development.

This summer's release brings this on-chip debugging capability to an avalance of [new uniprocessor and multicore processors from Freescale, Intel, Broadcom, and Texas Instruments] and puts Wind River in a great position to help engineers tackle their debugging challenges so that they can enjoy more of their summers.


P.S. If you have an on-chip debugging story that you want to share, I would be glad to hear it.

[You can follow me on Twitter - enwafor]

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>