Bit Rot Amnesia

By Mike Deliman

deliman_lg.jpgSpirit Rover has been having a number of problems recently, anomalous behaviors, losing contact with Earth, even forgetting commands.  Amnesia has been one of the larger concerns.

At the heart of this issue is a bank of flash memory.  Flash pretty much works by saturating cells to "1"s, then writing the cells that need to be 0's.  "quantum wells" hold the charge for later recall.  Since this is all done with minute electrical charges, one would have to wonder how long it could last.

Over time, the charge in the "1's" cells is bound to leak.  High energy particles from the Sun and other sources may also strike parts of the flash, inducing charges, corrupting the contents of the flash.  Over time, some cells may flip from 0 to 1, etc, or become "indeterminate", reading 0 some times, and 1 others.  If such degradation was affecting any of the data structures or other rarely-written data, it could contribute to an apparent loss of data.  Even though the file system in use performs wear leveling (a process to "spread usage out" in a bank of flash memory), the data being preserved must be written for wear-leveling to come into play.  It is possible that some of the configuration and directory structure data may have been degraded over time.  This degradation is called "bit rot".

The only way to fight "bit rot" is to re-write the data with high confidence.  The best way to do this is to "erase" the entire bank of flash, and install a new file system.  The reformat command was issued to do this job.

Erasing and formatting a flash drive is among the most power intensive operations the rover has on Mars.  It is not surprising that it took 3 days for the results of the process.  Day one we format the drive, and get results of that.  Day two we do some test exercises and store the data in flash.  Day 3 we attempt to recover that data.

I guess now we have one solid metric for how long data can survive in a flash file system on Mars!

Congratulations to the MER team, on recovering saved telemetry from the flash drive.