What exactly are you testing, and how?

"What exactly are you testing? How are you testing it?"

I'm sure that these are amongst the questions that product, business, and technical managers across the embedded software industry have asked themselves on several occasions throughout their careers. I know I have.

The importance of testing embedded software isn't new. In my "youth", I remember our VP of Product Development circulating a white paper that discussed the devastating impact resulting from the deployment of an untested "simple" patch to some switching software and declaring that "this cannot happen to us". I also recall reading and feeling sad about a software bug in a medical radiation instrument that when combined with operator error resulted in lethal doses of radiation being administered to patients.

There is no debate that software quality assurance is serious business. But how seriously are we really taking it, particularly in non-safety critical software applications? Survey data continues to tell us that we're spending upwards of 25% of our project budgets testing and fixing our software, yet we're still not satisfied that our testing practices are effective. Hmmm...

Transparency and visibility into what is being tested is a concern. One of my colleagues blogged about this not too long ago. Confidence that you've covered all your usage different scenarios is another big concern. I can tell you that when I run software that has "encountered an unexpected exception" and I lose work, I get irate and think about other software options.
[editor's note: I thought about including a link to a PC vs. Mac commercial here, but thought against it. Sort of...]

I have invested in partnering with Integrated Processing Limited (IPL) to provide an extension to Wind River Workbench that provides developers with a professional unit testing solution for C/C++ development. This week we are announcing and launching IPL Cantata++ version 5.3.2. Cantata++ provides software developers with unit testing capabilities for Wind River's run-times - including VxWorks and Wind River Linux. Cantata++ 5.3.2 introduces a cool new capability called "robustness testing" that is useful for automating the generation test cases - useful for reducing the likelihood of those "unexpected exceptions".

We are pretty jazzed about this product release. Several of the enhancements made should make it easier to integrate Cantata++ into your environment - including extending Cantata++ to support proprietary and 3rd party embedded operating systems. I anticipate that as we move further into the multicore era - the percentage of multi-OS systems will continue to increase as our customers look to consolidate the number of processors in their systems. One of our goals at Wind River is to provide our clients with a choice of development tools to help with these usage scenarios.

To learn more about our unit testing solutions, please come and check out our webinar on The Benefits of Effective Unit Testing for Embedded Development this Thursday November 5th. There still some great seats available. If we don't see you there, please feel free to "start the conversation" right here.