Mike Deliman

Wind River Webinar: Q and A

Hello All,

last week as some of you know I was the featured presenter / presentation for a Webinar.   (you may have problems watching that with firefox...) .  During the course of the webinar, we were asked a number of questions, and we ran out of time...

Here are some of the questions and answers we couldn't get to.

--
Q: What is the biggest advantage of Vxwork over other real-time operation system?
A:  VxWorks is the most well-deployed and well-used commercial realtime OS in the world.  Because of this, the OS is more well-tested than any other commercial RTOS, for this reason I would say the maturity of VxWorks is perhaps it's greatest advantage.
--
Q:have you ever patched the OS or only the application sw?
A: In most cases, only application code is updated in space, though it is possible to patch  or replace even the bootrom code in many of the space robots.
--
Q:Can VXWorks 'replace' itself in case of malfunction?
A: I'm not certain what "replace" means in this context; some of our customers have invented ways to detect bad RAM locations, map around those locations, and load vxWorks to the remaining RAM. 
--
Q:How do you ensure that the RTOS does not crash? Even due to a simple NULL-pointer access? Do you have any built-in Crash Recovery mechanisms in WindRiver for Space systems?
A: While debugging with newer versions of VxWorks it is possible to use the MMU to trap accesses to, for instance, the 0-page, to catch accesses to uninitialized pointers, etc.  Once code is sufficiently  debugged, the MMU may be disabled if desired, or the product may be deployed with the protection enabled.  As far as crash-recovery systems, most of the systems in-flight have created health maintenance and monitoring systems, and event logging systems.  Wind River has learned from this, and our newer OS releases have support for configurable event logs and health monitoring.
--
Q:Do you customize Vx Works for individual customers ?
A: We have services experts available to help with everything from the initial installation, to implementation of the entire product, including modifying vxWorks for a particular project.
--
Q:Which the CPU platforms are apted for VxWorks? How to get more info about VXWorks and its using?
A: VxWorks is available for many PowerPC, MIPS, ARM, Xscale, and other CPU types.  Wind River has offices world wide.  Please check at WWW.Windriver.com for office locations.
--
Q: I know you have a trial version of vxworks, but for a person who want to learn it, it expires quick. do you have a slim version with no expiration?
A: We do not have a "slim" version.
--
Q:  Is there a Webinar that spotlights VxWorks? A Demo?
A:  Demos are available of all of our products, pleas contact your local sales office.
--
Q: how do you update your software on space robots?
A: This is highly dependent on the hardware and hardware capabilities used to implement the robots, so unfortunately our customers must usually invent the right methods.
--
Q: AI and VX Works.. Any efforts to embed supporting AI as a native support on VX Works?
A: Though AI systems and some AI capabilities have been implemented using VxWorks, I am not aware of any efforts to embed more than rudimentary AI functionality along with VxWorks; Stardust, DS1, and the Mars Exploration Rovers are the best examples I can think of that incorporated any degree of AI.  Wind River is not planning on adding any AI capabilities to VxWorks (or Linux) at this time.
--
Q: how do you know when to time out communication to a mars rover when there is so much delay and interference?
A: That is left to the folks who implement the radio protocols used by the Deep Space Network to communicate with all the probes/robots/satellites in deep space.  They are experts with communications in deep space.  I expect sometime in the near future that this will all change with Delay Tolerant Networking and the implementation of the InterPlanetary Internet.
--
Q: Is VxWorks migrating from support of ASIC hardware to FPGA's?
A: VxWorks runs on a variety of platforms including COTS boards, ASIC, and FPGA based designs.
--
Q: All application using embedded controller or PCs?
A: Many manufacturers of COTS computer boards for VME, PCI, or cPCI supply VxWorks BSPs for their boards, and Wind River Systems supports BSPs for several COTS boards directly.  I hope this answers the question.
--
Q: Do you see an increase in the percentage of embedded applications that use some form of Linux vs others like pure VxWorks (i.e. not including VxWorks Linux)?
A: To be clear: VxWorks is NOT Linux, the two are not even remotely related to each-other; vxWorks pre-dates Linux by... years.  Likewise, Linux did not evolve from VxWorks.  They are similar in some respects (Posix, networking support, etc), but they are not even "kissing cousins".  Wind River does have our own versions of Linux available along with VxWorks.
Though I have seen an increased presence of Linux in the embedded arena, and an increased presence in the development and testing phases of even software for space applications, I have not seen Linux promoted to controlling a mission (e.g. the primary flight computer) yet.
--
Q: What problems are unique and interesting to  underwater implementations of VxWorks like those faced by MBARI?
A: I wish I had a contact at MBARI for you to ask!  As far as I know, most issues unique to the situation would be shared by all submersible vehicles, and MBARI has excellent experience in dealing with submersibles and software for underwater applications. Once the mechanical issues of sealing out the environment are taken care of, weather it's space or deep-sea, the rest becomes implementing software to control your devices, debugging, and deployment.
--
Q: [referring to an earlier question] Additional details on my earlier question on RTOS decision making ... The specific application is for an RTOS decision to be made for new instrumentation used in human spaceflight ... that is where do I start?
A: I would imagine you would need to start with the specifications for your deliverables: what kinds of certifications you will need, if any, and what kind of constraints the computer needs to operate under.  For instance, will this be a deep-spacee project requiring rad-hard hardware, or will rad-tolerant hardware suffice?  Will you need FAA or military certification, or none at all?  Wind River is happy to discuss the software packages we have and how they may be applied to your project.
--
Q: which vxworks versions have flown in the past and how customized were they ? (components)
A: VxWorks 5.2, 5.3.1, 5.3.1 MER Edition, 5.5.1, and I think 6.2 have all flown in space.  Other versions have flown in military aircraft, etc.  The most customized component of VxWorks I believe would be the DosFS file system on the MER rovers.  The folks at JPL made some great improvements after the SOL18 issue. 

For the most part, we strive to keep the releases "as close as possible" to the standard releases in order to facilitate technical support and software maintainability.  Newer radiation hardened chips are available that are very similar to commercial parts, like standard PowerPC or Sparc chips, and these newer chips run the same (current) versions of vxWorks as everyone else does.  This allows the customer to use our standard technical support for many issues, making experts more available for all issues.
--
Q: There were flash managment issues on MER and the Polar Lander. Could you explain the issue?
A: The flash management issues on MER "A" Spirit was actually more a RAM management  issue combined with a debug feature, precipitated from a "feature" of the DOS file system.  I am not intimate with any problems experienced on Mars Phoenix Lander, but the last I heard MPL's experts have identified a possible application problem.  Given when I'd heard this, I'd expect they may already have tested and sent-up a fix.
--
Q:Is VxWorks already been ported to RAD750 or Leon3FT ? Do you support academic R&D with easy to access software or anything other than that ? Thanks!
A: Yes - Rad750 BSP is supported by BAE Systems, from Wind River's perspective it's pretty much a "generic" PowerPC 750 and uses standard software and tools.  BSPs are available for VxWorks versions 5.x and 6.x, I believe 6.4 is available and a BSP for 6.6 will be available soon.

Wind River does not directly support VxWorks on Leon, but the manufacturer does have a solution with VxWorks 6.x available (I find this very fascinating and would love to "play" with it sometime).  :)

Wind River does have a University Program, contact your local Wind River sales office for details.
--
Q: How many versions/levels are there of VxWorks, and, how do you choose a version of VxWorks for example a Mars rover?
A: VxWorks has been around for a while, well over 20 years.  When I first saw it, the version was 4.0, and there was one version of vxWorks that ran on top of other companies kernels.  Now there are various versions of VxWorks for specific markets, and platforms available to help tailor VxWorks for specific usage.  I would always recommend using the latest version available for your hardware platform that satisfies the needs of your program.

This was the first webinar I've ever presented.  I hope you enjoyed it as much as I did. 

What's in a Kernel?

Recently my cohorts Paul Parkinson and Doug Gaff have been making a bit of noise about changes in the software industry - evolution of software, and specialized software for military and commercial avionics,and other applications that have strict time and security requirements.  Aside from requiring strict adherence to standards, critical systems software - especially medical or flight related - have an increasing litany of certification inspections it must pass.  In the secure-systems markets the same trend is evident.  Each of these markets are starting to require time- and space- critical systems.

Let me define time and space critical. 

Time critical is pretty straight forward - anything that has to be done within deadlines, is time critical. Sometimes time-critical functions are also periodic in nature, requiring periodic service with deadlines.

Space-critical is a bit more slippery.  This is about RAM, but not just "how much".  I'm talking about space in terms of RAM that other processes are protected from, and RAM that is protected from this process, and even memory shared between processes.

There are a lot of examples of time critical periodic applications.  Any aspect of handling telemetry - course, attitude, velocity, housekeeping (checking fluid levels, etc) - has time critical periodic attributes.   It's easy to think of time-critical tasks.

Space-critical is a little more elusive to both define and decide where to use, especially in embedded and real-time applications.  The answer to time-critical is somewhere between having fast enough hardware, efficient enough software, and proper scheduling. Protecting memory areas on a process-by-process basis has excellent benefits, but those benefits usually come at a cost of slower execution.    Sure, you never want your stack to be overflowed, your text to be zero-cleared, or your exception table to be  the victim of an uninitialized data pointer, but when is the trade-off in execution speed penalties worth the  overhead of enabling such protection?  The answer has more to do with security than with protection from poorly-debugged code - the separation, multiplexing, encoding, decoding and proper sorting of information streams.  There may be situations where every function of your application must have it's own data spaces because of the nature of the data being handled.  In fact, this kind of design is becoming more common.

It seems like it would take a fantastic amount of processing power to be able to do what's described above - guarantee on-time scheduling to handle deadlines, and mix-in varying levels of RAM-security, and still provide all the services and infrastructure necessary from an O.S. to build effective applications on-top of.  Even though processors are being created now with incredible horsepower, it wouldn't seem like they could keep up with much of this kind of work.  All that partition scheduling and context swapping would bog down the best of processors if it had more than just a few partitions and contexts to handle!

...or would it?

Come on down to the Regional Devlopers Conference in Manhattan Beach (Los Angeles) on May 24th, and we'll talk about it.  :-) See you there!

Test as you... surprise!

There's an old adage in the world of flight - space flight, or otherwise.

"Test as you fly, fly as you test."

It's pretty short and sweet and straight-forward.  Don't fly what you haven't tested.  Test exactly the ways you expect to fly.

In a big, round, wonderful world, it seems "funny" when we hear about problems related to living here.  Problems like aircraft "flipping over" when they crossed the equator because they were adjusting to negative latitude - the coder hadn't thought far enough ahead to think about crossing the equator.

This is the kind of problem that should be found in test, simulation, or in a thorough validation of design and implementation.  The kind of validation that should be done with "other eyes" - not the eyes of the implementing entity.  It's far better to find these kinds of problems before you're in the air, and lives are depending on everything to be Reliable.

That's why I was surprised to read today about problems with the F22, apparently crossing the International Date Line caused the computers to shut-down.