Medical

Open source in Medical Devices: Part of the Cure or Part of the Disease?

By Ido Sarig

The Economist recently published a very interesting article on the merits of open source in medical device development, which raises some questions and sparks an important discussion. 

In general, it is more common to see open source adoption in non-regulated industries. However, we are seeing it more and more in regulated industries primarily because it encourages rapid innovation at far lower costs.  That said, it is important to caution those in the regulated industries (aerospace & defense, medical, industrial automation, energy, automotive) towards embracing open source concepts without investing in lifecycle management tools to support the building of safe and effective devices.  [Proprietary software on the other hand, is a better fit for companies that design devices in environments where changes are few and far in between. Proprietary software also enables better control of the ecosystem, the lower the stack you go.]

As the Economist puts it, “exposing a design results in safe products.”  Maybe…Designing a device or ecosystem with safety in mind, results in safer products, hence the need for “design for safety.”  The FDA mandates a safe and effective design for any devices that gets approved in the US market -- this requires companies to have better control of their ecosystem.  Done proactively, this is an efficient method for product development; however, done reactively, companies incur a lot of costs and overheads trying to make open source fit.

The basic premise of the Economist article, and apparently the FDA’s thinking is that the more eyes on the code, the more bugs will be detected. But that premise is questionable.  A recent study commissioned by the U.S. Department of Homeland Security’s Scan project found that “Where codebases were of similar size, open source code quality was pretty much on par with proprietary code quality.”  For example, the open-source Linux codebase, which is about 7 million lines of code averaged 0.62 per 1000 lines of code (KLOC), while proprietary code bases that averaged around 7.5 million lines of code had 0.64 defects per KLOC. So it seems that in large, complex code bases, such as medical devices, open source is not a panacea, and rigorous testing still has to be performed. One of the most challenging aspects of thoroughly testing medical devices is testing all the error handling code and the exception handling code. This code often makes up 50% or more of the overall code in embedded software, yet is the least tested- because the error conditions themselves are difficult to set up or simulate.

With Wind River Test Management’s Sensorpoint technology, one can easily inject faults directly into fully optimized, production-ready code as part of the test process, making it easier to achieve the complete code coverage required by the FDA for regulatory approval.  In fact, one of our customers was able to reduce the amount of time required to get their latest ultrasonic surgical  knife approved for use by the FDA by 15%, using Test Management’s Sensorpoints.

Another risk factor associated with open source is the fact that contributions come from multiple sources, whose identity is not fully known, exposing the device to the possibility that malware may be incorporated into the code base. As we’ve seen, having the code base exposed to more eyes does not, in and of itself guarantee lower defect rates, so it stands to reason that it does not dramatically reduce the incidence of security flaws.  The problem is exacerbated by the fact that open source libraries are often download from a repository that does not have the latest version of the components.  Research by Sonatype and Aspect Security released earlier this year, confirms this:  looking at the most popular open source Java frameworks and security libraries, they found that 26 percent had known vulnerabilities, and 41 percent were older versions of the components.  Testing for security, incorporating techniques such as Fuzz testing, can help reduce that risk. With Fuzz testing incorporated into Wind River Test Management, even non-security testing professionals can make security testing a part of their normal QA routine.

The bottom line is that enabling open source with industry-leading embedded lifecycle tools will be a happy medium for companies that don’t want to be left behind in technology advances during the life of their devices. If we can extend better healthcare to the masses through affordable open source platforms, we are all for it. Do connect with us to have a discussion on how embedded devices can embrace open source strategies by the right adoption of tools and processes.

How Are You Reducing Radiation?

By Pete MacKay

A few years back the FDA launched an initiative aimed at reducing radiation exposure in patients undergoing CT scans.  Clearly there are concerns with exposing the human body to ionizing radiation, and due to the prevalence and popularity of these imaging techniques the FDA feels they are in a position to ensure proactive steps are taken to minimize exposure.  They cite studies done in 2009 showing this total exposure to have nearly doubled in two decades in the US population.

Several factors come into play with cause and effect here, and the FDA is exploring several different perspectives.  Preventative measures start with educating patients, clinicians, and technicians, and altering approaches to therapy planning and the decision process around imaging studies.  The emergence of highly connected health enterprises (and society in general) enables a proposal to track cumulative dosage using tools like a national dosage registry and a patient medical imaging record – who knows, maybe just like Fido runs around with a chip in his neck we might all someday be implanted with a dosimeter along with our medical record?  Kidding aside, the obvious place to start with dosage reduction is with the device manufacturers themselves.

Not to carbon-date myself but I started in image processing back when an ALU was a very large (and warm) chip and processing functions comprised entire boards.  We needed big racks to accomplish a fraction of what’s now done in DSPs, GPUs, or even just coprocessor pipelines.  When I later worked for ADAC Laboratories in Milpitas, California – pioneers in nuclear medicine – I remember developing a network proxy (largely on my own time) for some of the embedded functionality of the piece I was working on.  We were building a transmissive system where a patient is radiated by a shuttered source on a robotic arm opposite sensors, and this prototype development took place in a lead-lined “safe” room.  I wrote code to run this machine from my desk so I wouldn’t have to spend time near the radiation.  One of our consultants chided me for this; he grabbed a Geiger counter and took four readings, the first two in the “safe” room with a barium source shuttered and un-shuttered, the next at my desk, and the last one in the parking lot outside.  Imagine my surprise to learn I did all that work only to find the lowest exposure was actually inside the safe room – even with a glowing source!

Fast forward to today… medical imaging device manufacturers are meeting dosage reduction goals with more sophisticated image processing algorithms that require massive processing power.  Engineering organizations now face cost reduction, tightened schedule expectations, and the demands of building extremely powerful yet flexible hardware/software architectures in a highly regulated environment.  Wind River can help meet these challenges with a range of integrated tools from Workbench to Wind River Test Management, performance-tuned compilers and libraries, advanced multi-core and virtualization technologies like Wind River Hypervisor, and systems expertise from Wind River Professional Services.  Wind River has helped manufacturers get all classes of medical devices approved for nearly the past three decades, and we can help meet radiation reduction imperatives in this one now.

 

For additional information from Wind River, visit us on Facebook.

Hacking Insulin Pumps for Fun and Profit

By Ido Sarig

Last month, I had the opportunity to take part in the Amphion Medical Forum in Minneapolis, where the theme was security challenges facing Medical devices. Amphion is a forum that brings together thought leaders from academia, business, government and technology, which was founded to provide a medium for these visionaries to define solutions to some of the issues, such as safety and security, which affect the new interconnected economy, the “internet of things”.

I participated in a panel that discussed the challenges and risks posted by the latest generation of wireless medical devices, and how proper Fuzz Testing, such as the capability offered by Wind River Test Management in conjunction with our partner Codenomicon, can help alleviate some of that risk.

What really grabbed my attention was the opening session which consisted of a demo of hacking someone's Insulin Pump. It recounted a session given by Jay Radcliffe at this year’s Black Hat conference in Las Vegas. It was an enlightening session showing how vulnerable medical devices are to remote attacks carried out wirelessly, and honed in on the fact that device manufactures have to address two conflicting design goals  - ease of use and security. Features added to facilitate programming and updates of the software, such as wireless communication and remote controls, open up new attack vectors which utilize these interfaces, and enable hackers to penetrate the system using venues previously unavailable to them.

But as interesting as this presentation was, a lingering question hung in the air - Why would anyone want to hack an insulin pump? And if no-one has an incentive to do this, perhaps we should not be concerned about these security vulnerabilities?

Radcliffe did it for what could possibly be termed “fun”- he is a security researcher, with both a professional interest in seeing if it could be done, as well as personal interest – as he has such a pump implanted in his body. But could somebody attempt to do something like this for financial gain? Sadly, the answer is yes. There are several such scenarios – starting with the ominous blackmail scenario: An unethical hacker could find such an exploit, and then blackmail the manufacturer, asking for huge sums of money in exchange for divulging the exploit’s details and keeping quiet about it. Or he could sell the exploit to a competing company, figuring out they would want to publicize a vulnerability in their competitor’s devices. But it doesn’t even have to be that sinister to work. Suppose an unethical hacker finds vulnerability in a device made by a public company. All they have to do is sell short that company’s stock, then publicize the vulnerability and wait for the inevitable stock price decline that comes after the media frenzy, to make a pile of cash.

The bottom line is that neglecting to thoroughly test your medical device for security vulnerabilities,  thinking that no one will go through the effort required to hack for lack of incentive it is just plain wrong. If there is money to be made – and there clearly is – someone will do it.

For additional information from Wind River, visit http://www.facebook.com/WindRiverSystems.   

A Medical Device Platform: Beyond the Operating System

By Bill Graham

With the release of the new Wind River Platform for Medical Devices it's a good time to point out the breadth of products and services that make up a modern embedded solution suite. In fact, our medical solutions go well beyond the products that make up the Platform. The Platform for Medical Devices provides the main components: VxWorks RTOS, middleware, networking, development tools and BSPs. The following block diagram shows the highlights of the components in the platform:

For medical devices there are likely requirements for other technologies such graphics, multi-OS and partitioning (via embedded virtualization), embedded Linux, Android, JTAG debugging, testing tools and system simulation. Equally important are the professional services, technical support and long term vendor relationships. Our customers are looking for more than just technology they need the know-how to help them use the products, to assist with regulatory approvals, customization, technical support and training. So, an overall medical device solution is much more than the Platform for Medical Devices alone:

 

The release of the new Platform for Medical Devices is great news, but it's worth pointing out Wind River's long history of success in the medical market -- we bring experience and know-how to the table, plus we have a breadth of solutions to offer the medical device developer. 

When embedded software testing is a life or death issue

By Ido Sarig

When I was in high school, there was a popular TV show called “Moral Dilemma” that aired in my native country. It featured a panel of professionals such asdoctors and lawyers who discussed “life or death” issues related to their daily work:  Would you represent and defend a client charged with murder after he confided in you that he indeed killed his victim? A kidney has become available for transplant – do you operate on the rich, 85-year-old patient who promised to bequeath a million dollars toward the creation of an organ bank at your hospital, or provide the kidney to the teenager who was just accepted into the country’s most prestigious college?

The show portrayed these professions as dealing with the most important moral issues imaginable, and by implication, relegated other professions to a lower rung on the ladder. As someone growing up from a family of engineers, I knew engineers can have significant roles to play and I definitely had respect for other professions not represented on this TV show. However, it did get me to thinking: Were there also life or death issues in software?

Fast forward 30 years, and I know that there are indeed life or death scenarios in the world of engineering and software. Embedded software is a vital element in the majority of today's sophisticated medical devices—and when it fails, disastrous results follow.

The other day, I was reading  about a (yet another!) FDA Class I recall of infusion pumps.  This particular recall was ordered before any malfunction actually impacted a patient, but previous instances were not so fortunate. Problems with infusion pumps have been linked to more than 56,000 reported complications over the past five years, including at least 500 deaths.

The irony (if you can call it that) in this particular recall is that these specific infusion systems feature a wireless radio enabling remote programming that was being marketed as a patient-safety solution to manual programming errors - yet it was precisely this wireless radio component which failed and triggered the recall. According to the FDA, when used in a wireless network environment that utilizes Temporal Key Integrity Protocol (TKIP) authentication, a particular brand of compact flash can potentially induce a memory leak that can cause the Management Processor to become non-responsive. This causes normal operation to stop, preventing the pump from delivering the medicine with no visual error warning to alert the user that the pump is not working.

How does this happen? A key element to consider in this situation could be insufficient medical device testing.  As device software grows in functionality and complexity, the process to test and verify that it works correctly becomes increasingly complex and difficult. Medical devices manufacturers have a nearly endless matrix of combinations that require testing.  Making matters worse, market dynamics force device manufactures to shrink their product development cycles, leaving less time than ever for comprehensive testing programs.

In this case, it would have been critical to test not only the Motorola Compact flash which failed vs. some-other-vendor's flash, but also the use of TKIP vs. other authentication protocols [the device supports at least 4: WEP (40 and 128 bit keys), WPA-PSK or WPA2-PSK), 3 different wireless LAN standards (802.11 a/b/g), and 2 ways of obtaining IP addresses (Static or Dynamic (DHCP)]. So, there are 24 different configurations for each type of compact flash just to test basic wireless connectivity issues, and this is before we even get the functional testing of the various infusion modes, dosage programming modes, error code checking etc…

And this is where the moral issues come into play. Exhaustively testing every possible combination of software and hardware would likely take many months, and would have to be repeated every time a new software version is released by the developers, which happens quite frequently during the development cycle.

The manufacturer does not operate in a vacuum: there will always bemarket pressures and competitors releasing new models as well. But is it right to release a medical device that has not been thoroughly tested in order to better compete? And how do we know enough testing has been done to sufficiently cover all the complex functionality that goes into modern infusion pumps?

Luckily, we can reduce some of this uncertainty by using tools like Wind River Test Management  – which provide us with clear indication of which parts of our code has not been tested at all, help us focus our testing efforts on those risky parts of the code, and provide management with insight into the quality of the device they are about to ship.

I recently discussed some of these issues at a Webinar titled. “Medical Device Software Testing: A Life-and-Death Challenge.” You can view a recording of it here, and learn what you can do today to improve the quality of software in your medical devices.