Wind River Blog Network

Detecting security problems – using static analysis to catch them early and less expensively

By Bill Graham

In my previous post I discussed the potential benefits in quality and costs that static analysis brings to software development. In addition to common coding errors, many of the bugs found by static analysis are potential security defects as well.  Buffer overflow, OS command injection, unrestricted string format and integer overflows are among the top 25 most dangerous security coding defects (according to the Common Weakness Enumeration (CWS) from the MITRE organization). These types of defects are common in C and C++ and are dangerous to correct operation in general but also pose significant security threats – often because the right exploit can lead to arbitrary code execution on the target. Once an attacker can execute code on your device, they can gain complete control, which might include reflashing the firmware, installing malware or rewriting the command and control software.

Continue reading >>

Using Static Analysis to Improve Product Quality, Earlier and Cheaper

By Bill Graham

Fixing bugs is expensive. Fixing bugs is more expensive the later you leave them, in fact, its been shown to cost a magnitude higher with each major phase of development. The famous defect cost chart from Capers Jones shows the cost of a bug going from $25 at the coding phase to $16,000 in development.  Not only that, but most of the bugs are introduced into the system at this stage yet very little of them are found. Logic says we should fix them earlier to save money.

Continue reading >>

Hacking Insulin Pumps for Fun and Profit

By Ido Sarig

Last month, I had the opportunity to take part in the Amphion Medical Forum in Minneapolis, where the theme was security challenges facing Medical devices. Amphion is a forum that brings together thought leaders from academia, business, government and technology, which was founded to provide a medium for these visionaries to define solutions to some of the issues, such as safety and security, which affect the new interconnected economy, the “internet of things”.

I participated in a panel that discussed the challenges and risks posted by the latest generation of wireless medical devices, and how proper Fuzz Testing, such as the capability offered by Wind River Test Management in conjunction with our partner Codenomicon, can help alleviate some of that risk.

Continue reading >>

Secure Mobility Message Resonates at MILCOM 2011

By Chip Downing

MILCOM 2011 last week in Baltimore, MD was not only educational, but it was a watershed event.   Last year, it was a “coming out” party for custom Android devices; this year however, was the year where many companies are now using COTS hardware and software to their fullest extent, along the entire supply chain, from core to edge and user devices.   I believe this COTS adoption is driven by two powerful forces – the exploding capabilities of commercial tablet devices, and global military budget pressures.

The vastly increasing power and capability of commercially available mobile communications devices is simply amazing. Just a few years ago, almost everything in this military communications space was purpose built, and most commercial products were simply inadequate to address the rugged and complex system demands of military communications systems.   As seen at MILCOM 2011, this has officially changed -- COTS end-to-end communications domains rule.

So if all of these MILCOM vendors are not making customized hardware and software, what are they doing?   Wind River had over 500 people visit its booth at MILCOM this year, and most of the discussions I had with both suppliers and end users at the show were about increasing device security and leveraging high volume COTS components in demanding deployment scenarios.

Continue reading >>

Improving Embedded Security: Proper Runtime Selection

By Bill Graham

Selection of secure components for an embedded system is key to a secure system. Leveraging a secure RTOS, middleware, virtualization and tools significantly reduces the effort and development costs. Moreover, there are additional benefits from using commercial-off-the-shelf (COTS) software components over Roll Your Own (RYO) code or self-ported and maintained open source code. Some of the COTS components worth considering include:

Continue reading >>

Test Management 4.0: Reduce Risk by Understanding the Impact of Change

By Ido Sarig

Big day today, we just announced the release of our latest version of Wind River Test Management, WRTM 4.0, the culmination of several years of hard work - kudos to our engineering team!

In the coming weeks, I will be covering many of the new features of WRTM 4.0 in detail in a series of separate posts, but today I wanted to give you an overview of some of the new things you can look forward to.

I’ll start by sharing a quote from Wind River customer GE Healthcare on how Test Management is helping their business -- a real testament to the value that Test Management delivers.

"Wind River Test Management has tremendously helped us improve the productivity of our firmware testing team, significantly reducing the overall development cycle time. And although we already had more than 20 tools as part of our test and development process, test plans and test results were still mostly paper based.  Wind River Test Management easily integrates with our existing solutions, such as HP Quality Center, to provide a complete automated regression testing suite running tests remotely on the target and collecting valuable metrics for tracking test progress and software quality. GE Healthcare operates on a ‘zero defects’ policy and Wind River helps ensure that this policy is met much faster than before."

Continue reading >>

Testing for Security

By Ido Sarig

Last summer was a watershed event for security-consciousness in the embedded systems world: Stuxnet, a highly sophisticated worm exploited no fewer than 4 zero day vulnerabilities in Windows in order to attack a specific Siemens PLC and its associated SCADA system. The target was reportedly the Iranian nuclear facilities at Natanz, where uranium-enrichment centrifuges were taken out of commission by the worm’s malicious payload. It was perhaps not the first, but certainly the most well-publicized successful attack on critical infrastructure systems.

The software security industry has been discussing such an attack for years, mostly as a theoretical possibility – but now it appears this is no longer the stuff that Hollywood scripts are made of - it is all too real.

Continue reading >>

When embedded software testing is a life or death issue

By Ido Sarig

When I was in high school, there was a popular TV show called “Moral Dilemma” that aired in my native country. It featured a panel of professionals such asdoctors and lawyers who discussed “life or death” issues related to their daily work:  Would you represent and defend a client charged with murder after he confided in you that he indeed killed his victim? A kidney has become available for transplant – do you operate on the rich, 85-year-old patient who promised to bequeath a million dollars toward the creation of an organ bank at your hospital, or provide the kidney to the teenager who was just accepted into the country’s most prestigious college?

The show portrayed these professions as dealing with the most important moral issues imaginable, and by implication, relegated other professions to a lower rung on the ladder. As someone growing up from a family of engineers, I knew engineers can have significant roles to play and I definitely had respect for other professions not represented on this TV show. However, it did get me to thinking: Were there also life or death issues in software?

Fast forward 30 years, and I know that there are indeed life or death scenarios in the world of engineering and software. Embedded software is a vital element in the majority of today's sophisticated medical devices—and when it fails, disastrous results follow.

Continue Reading >>

 

Integration and Testing the Integration

By Jakob Engblom

The US National Highway Traffic Safety Administration (NHTSA) recently released a deep report into last year's issue with "unintended acceleration" on certain Toyota cars. They actually employed a team from NASA who analyzed the throttle control software using a wide range of cutting-edge tools. Reading their report gives a good idea for how embedded control software is developed, and the challenges inherent in validating it.

The conclusion is that the software is not a likely cause of the problems. The real meat of the report (and its appendixes) is just how this conclusions is reached, and what it says about the tools that could be applied to develop embedded software with greater confidence about its correctness.

Continue Reading >>

 

Being Helpful or Simply Correct?

By Jakob Engblom

The other day, I spent some time getting a new operating system up on one of our Simics virtual platforms. The platform is stable, the hardware is shipping and it is being used with the very software I was setting up. However, as the operating system was booting, I got quite a lot of warnings from Simics about incorrect hardware settings. The operating system still worked, however, so why did we see all these warnings? Actually, when is it prudent to issue warnings to a user about suspicious uses of hardware? When building Simics models, we want them to be helpful and offer warnings as much as possible - especially for cases where some uses might be considered technically "correct" but are actually somewhat suspicious.

A virtual platform like Simics can print warnings to the simulation console when software performs operations that are not legal according to the hardware specification. Typical examples are writing to offsets in device register banks where there are no registers, changing the value of bits which are marked as "reserved", or writing illegal values to configuration registers.

Continue Reading >>

 

What the LA Lakers and Apple iPhone can teach us about software engineering

By Ido Sarig

Half way into the 2010-2011 NBA basketball season, the Pacific division leading LA Lakers had an embarrassing loss to the bottom-dwelling Memphis Grizzlies. The LA Times described the performance of several of the players at the Sunday, Jan.2, 2011 game as “sleep walking” – which turned out to be not far from the truth!

Three Laker players had reported oversleeping the night before the game, because the alarm clocks on their iPhones malfunctioned that morning, part of a well publicized failure.

If this sounds familiar to iPhone users, it’s because it was a repeat of a nearly-identical episode, reported just two months earlier, when the iPhone alarm application failed to operate properly after the switch back from Daylight Saving Time to standard time.

What does this have to do with software engineering? It illustrates a well-known phenomenon in large-scale software applications, researched as early as the 1970s, when IBM performed a “distribution analysis of customer reported defects against their main commercial software applications” and were “surprised to find that defects were not randomly distributed through all of the modules of large applications” – or in plain English - some parts of the code are more buggy than others.

Continue Reading >>

 

Testing? That’s what customers are for!

By Ido Sarig

Many years ago, circa 1990, as I was starting out my professional career as a software developer with Mercury Interactive, I went on a customer call to a Silicon Valley-based software company, to present our new product for automated software testing. Apparently, our message did not get through, as the response we got at the end of our pitch from the company’s VP of engineering (who shall remain nameless) is the title of this post.

I don’t want to go as far as saying that this was the prevalent attitude in those days, but I also can’t honestly say it was atypical. Back then, many in the industry, especially in engineering roles, thought it was a reasonable process to develop an application, debug it as best you can, ship it, and then fix the remainder of the issues as they surface at customer sites. This, despite numerous studies showing it is much more costly to repair bugs found in the field than it is to find and fix them before shipping.

Continue Reading >>

Low-cost Android phones: Feature phones an endangered species?

By Chris Buerger

There is no doubt that there will always be a set of consumers that only want their mobile phone to be, well, a phone. Voice calling is likely to remain the ‘killer app’ for the majority of consumers for a long time to come, and there will remain a sizeable section of the mobile phone market that will cater to this demand.

However, the market change to smartphones is undeniable, even in the last bastions of feature phone-centered markets such as Japan and South Korea, Android-based smartphones are rapidly winning market share at the high end of the device spectrum. Now, a second wave of the Android smartphone revolution is happening in many countries, and I am excited to say that Wind River mobile software solutions for Android are right in the thick of things, helping to drive this new wave of devices.

Continue Reading >>

Physical or Virtual

By Jakob Engblom

When you explain what a virtual platform or full-system simulation solution like Wind River Simics is, a key part is that it runs the same software as a physical board or physical system. This does capture the core of the idea - but it does not mean that the virtual platform is only "just like the real thing".

Comparing virtual and physical systems is like comparing apples and apples, not apples and oranges: while apples are mostly interchangeable, they is certainly variation between them. Some apples are best for eating, some are better for making sauce, some are pie material, and some are best for fermenting cider. The type you select depends on what you want to cook. The difference between physical and virtual hardware is similar: they can be used as replacements for each other to some extent, but the connoisseur can make much better use of both by looking at the differences.

Continue Reading >>

Test Automation Meets Simulation

By Paul Henderson

I'm seeing increasing interest from many companies in using simulation environments with test automation systems to accelerate the testing process. Specifically, putting Wind River Test Management together with Wind River Simics is getting creative juices flowing in industry thought leaders.

Why? Well, development teams have started to realize the benefits of simulation systems for speeding and validating system and software design, and for accelerating software development and debug in advance of hardware availabilty. And even when hardware is available, systems like Simics provide tremendous access and control to speed analysis and diagnsotics.

Continue Reading >>

Transporting Bugs with Checkpoints

By Jakob Engblom

I have a paper about "Transporting Bugs with Checkpoints" to be presented at the S4D (System, Software, SoC and Silicon Debug) conference in Southampton, UK, on September 15 and 16, 2010. The core concept presented is to leverage Wind River Simics checkpointing to capture and move a bug from the bug reporter to the responsible developer. It is a fairly simple idea, but getting it to work efficiently does require that some things are done right.

Continue Reading >>

Wind River and IBM Attack Software Quality

By Paul Henderson

As I've mentioned before, we've been working with IBM Rational for some time around quality management automation. Both companies see the skyrocketing software content and architectural complexity in the embedded device market as creating a tipping point where companies will not be able to continue with business as usual.

Product development teams will need to take a more managed and automated approach to quality that spans across the lifecycle and access into the devices under test. This is particularly true in markets that require strict adherance to standards and compliance regulations.

We put together a joint whitepaper on this subject downloadable from here. And we are also having a joint web seminar next week on Tuesday Aug 31 at 2pm EDT. You can register for this event here.

Continue Reading >>

It’s Time for Testers to Step Up

By Paul Henderson

RTC Magazine recently published an article that I wrote called "Time to Rethink Software Testing for Embedded Devices". In it I describe some of the new techniques that are possible, and I believe necessary, to delivery high quality device software for embedded devices.

• When staying 'positive' doesn't pay
• Getting negative with white box testing
• Focusing on the 'deltas' with change-based test automation

Continue Reading >>

VxWorks = Secure

By Nikhil Chauhan

A recent report describes potential security vulnerabilities in devices running VxWorks. Researcher HD Moore claimed during a recent talk ( slides) that a quarter million devices accessible directly from the Internet were found to be vulnerable.

VxWorks has a very strong track record of offering secure products. However, we also realize that vulnerabilities can affect VxWorks, even if very infrequently. In those cases, Wind River will act quickly to address any issues. Regarding recent vulnerabilities, Wind River responded rapidly with patches and remediation steps in conjunction with a public announcement by the CERT Coordination Center on August 2, 2010. Once CERT notified Wind River, Wind River immediately assessed the alert and was instructed by CERT to release a synchronous public response. We're confident that our customers know that Wind River is committed to supporting its products with the highest quality and security standards.

Continue Reading >>

Test Driven Development Meets Continuous Integration

By Paul Henderson

In my last posting I mentioned I'd be running a webinar with James Grenning on Agile testing. James is a recognized expert and frequent speaker on the topic of software development and one of the original authors of the Manifesto for Agile Software Development.

We talked about the case for agility where today's embedded software projects are inevitably faced with changing requirements and market conditions that cause unplanned, mid-course corrections. The result is what went in is often not what was expected to come out. Testing folks are the tail trying to wag the dog as they try to test in quality at the end of the project.

Continue Reading >>

Agile Testing for Embedded Devices

By Paul Henderson

I am running a Web seminar on agile testing with James Grenning. James is a recognized expert and frequent speaker on the topic of software development. Founder of Renaissance Software Consulting, he provides training, coaching, and consulting to corporate and government clients worldwide. James is one of the original authors of the Manifesto for Agile Software Development and is presently leading efforts to introduce Agile development practices to the challenging world of embedded systems.

Iterative software development methods like Agile are being adopted by many software and information technology (IT) organizations across the industry. Quick design-develop-test cycles let teams respond better to changing requirements while providing timely feedback on features and quality. Agile methods help companies produce higher-quality software faster and at lower costs.

Continue Reading >>

Industry Investing in Better Device Runtime Visibility During Testing

By Paul Henderson

Here’s the final installment in my series about our embedded device software industry testing survey conducted in April-May 2010 with almost 900 respondents (see previous blog postings).  If you’d like a copy of the full report in pdf, please drop me an email at paul.henderson@windriver.com and I will send it to you.

In this section of the survey we asked participants about what test tools they use today and where they are investing in test automation. Given the high cost of product failure, accelerating complexity and reduced schedules the industry is turning to more test automation in 2010 to help address these problems. The top investment are moving to new tools that can help test teams and their management better understand how well they are testing, better focus their efforts on the areas needing testing, and reduce cycle time through more automation.

Continue Reading >>

The High Cost of Poor Quality – Brand, Market, Budget

By Paul Henderson

I’m continuing my series on our embedded device industry software testing survey conducted in April-May 2010 with almost 900 respondents (see previous blog posting).  If you’d like a copy of the full report in pdf, please drop me an email at paul.henderson@windriver.com and I will send it to you.

In this section of the survey we asked participants about how they measure the high cost of poor quality. Respondents told us that the true cost of poor quality is much higher than program budget. The majority of respondents showed that the true cost of poor quality is measure by damage to company brand and lost revenue due to missed market windows.

Continue Reading >>

Inadequate Management Visibility into Quality is Eroding Confidence

By Paul Henderson

Here’s the next installment in my series on our embedded device software industry testing survey conducted in April-May 2010 with almost 900 respondents (see previous blog postings).  If you’d like a copy of the full report in pdf, please drop me an email at paul.henderson@windriver.com and I will send it to you.

In this section of the survey we asked participants about how they measure software quality today, the metrics most often cited by survey respondents were reactive in nature such as tracking customer-reported failures and open defects rather than metrics that can help them prevent defects.

Continue Reading >>

Compressed Schedules Driving Shorter Testing & Defect Resolution Requirements

By Paul Henderson

Today I'm continuing my series on our embedded device software industry testing survey conducted in April-May 2010 with almost 900 respondents (see previous blog posting).  If you’d like a copy of the full report in pdf, please drop me an email at paul.henderson@windriver.comand I will send it to you.

In part 2 of the survey we asked about schedule compression and what affect that was having on the device testing cycle. A majority of survey participants reported that market conditions have forced them to shorten their development schedules by as much as 18 months.

Continue Reading >>

True Concurrency is Truly Different (Again)

By Jakob Engblom

A recent article at Ars Technica describes yet another security flaw in Windows. Nothing much new in that respect, but this is indeed an interesting attack in that it is enabled by using multicore hardware. It is not practical on a single processor, demonstrating once again how multicore is fundamentally different from multitasking on a single processor.

The attack targets software that hooks Windows kernel code to do additional work, such as anti-virus software. The idea is to get bad data into a hooked system call, by replacing input data after it has been validated by the hook but before it gets consumed by the kernel.

Continue Reading >>

A Crisis of Complexity – Industry Report on Growing Challenges in Embedded Testing

By Paul Henderson

I’ve been talking a lot with embedded device companies around the world over the last few years and I am hearing growing concerns about software testing. I’ve mentioned several of these concerns in previous blogs. I wanted to get more quantifiable data and get some feedback that could help us shape our products and services to help. So I decided to run a survey to gather important data from our community.

The focus of this survey was to gain a detailed snapshot of how executives, development managers, QA and test team leaders and other involved staff currently view the embedded device software quality test landscape. Recent changes, new challenges and strategies for managing them were of particular interest.  So I fielded a four-part survey to individuals who work for embedded products companies. In total, nearly 35,000 individuals in North America were invited to participate in the survey via emails. 

Continue Reading >>

What’s New in Wind River Test Management 3.3?

By Paul Henderson

Today we announced the latest version of Wind River Test Management, Release 3.3, our test automation system for monitoring, executing and managing embedded device software testing. Wind River Test Management lets teams optimally execute complex tests while dynamically gathering information from the production software under test as it is running, without requiring special pre-instrumented software builds. This approach allows teams to adopt new white-box test techniques that give testers visibility into the operation of the device and help them determine the thoroughness of the tests, quickly identify defects and performance bottlenecks, and focus efforts on sections of software that are most in need of testing.

Release 3.3 is a major new release that adds a number of significant new features. You can learn more and download several new whitepapers from www.windriver.com/products/test_management.

Continue Reading >>

ESC Silicon Valley

By Mark Hermeling

Looking forward to my trip to ESC Sillicon Valley next week. It is promising to be a busy show, especially since ESC is now combined with the Multicore expo. I just leafed through the agenda (in the form of a Nxtbook) and found a large number of sessions that I want to attend, experience show though that I'll probably be too busy talking to customer to attend sessions, which is a good problem to have of course.

I am hosting a 4 hour session (with several of my colleagues) on Multicore Demystified on Tuesday afternoon 2.30-7pm (there will be refresments!) in the Hilton Plaza Room. Do stop by either the session, or our booth at Multicore Expo for a chat if you want to brain storm about your next generation devices.

Continue Reading >>

Time to Get Positive about Negative Testing

By Paul Henderson

I spend a lot of time talking to device development and test groups and I continue to be surprised by a number of things. One is how little test automation is in place if you look across embedded device companies and industries.

But another surprise is how little companies are doing in the way of “negative” testing. By this I mean testing that tries to break the system, validate fault and exception handlers or otherwise force the device in to an unusual state or “edge condition”.

Continue Reading >>