Getting Code Coverage with LDRA Tools on Simics

A system development tool like Simics is rarely used completely on its own. Simics is usually brought in by companies and development teams who are already using a variety of tools and expect to keep doing so along with Simics. Also, Simics by itself cannot reasonably be expected to solve all problems for all people. Instead, we need to connect Simics with outside tools to provide an integrated solution.

In this blog post, Jay Thomas of LDRA and I will discuss one concrete example of such a tool integration. In particular, how Simics can be used together with the code coverage and certification tools from LDRA to perform code analysis on safety-critical software in a way that is more efficient than with most hardware platforms.

Jakob Engblom: We start with the basics. Can you introduce yourself?

Jay Thomas: My name is Jay Thomas and I am the Embedded Target Integration lead at LDRA. In my role I interface LDRA with a vast variety of embedded targets including a number of simulation environments. As such, I am always looking for tools to make our customers interaction with embedded targets easier, as it makes my life easier.

JE: What does LDRA do?

JT: LDRA provides a set of certification support tools. These tools start by connecting requirements to source code, then to test cases, execute these test cases on the target and provide results which are qualified and usable in a certified software development process. Some of these processes include DO178B/C, ISO 62304 and IEC61508.

JE: How important is code coverage analysis to a typical developer of a certified system?

JT: Code coverage is very important for several reasons. First, most safety standards explicitly call out a certain level of coverage. Second, coverage is invaluable in making sure your code does what it is supposed to do in all cases. Finally, when you connect those things, safety and making sure your code does what it is supposed to do with requirements traceability the resulting infrastructure has been shown to greatly increase product reliability and safety - especially in those cases where a failure can lead to a loss of life.

JE: Can you describe what you have done with Simics?

JT: With your help I added an interface layer to connect LDRA and Simics. This allows users to obtain qualified coverage from the Simics simulation environment and control Simics in order to execute unit tests. This allows customers to seamlessly execute unit tests on the simulated hardware without the typical delays of programming and data extraction that often impede their productivity.

JE: So I guess I better explain the Simics side of this.

In Simics, what we do is to add a small trace device to the target virtual hardware setup.  The trace device maps a one-byte location into the target memory map, and all bytes written to this location are sent to a file on the host.  The LDRA tool reads the file after the Simics run is finished and processes the results.

 

The target software instrumentation inserted by the LDRA tool performs writes to the trace device as the target software executes. The trace device is used instead of more common communications channels like serial ports or Ethernet. The driver for the trace device is very minimal, and requires far less software support in the target software stack. It is an easy thing to add to any Simics target, and it can even be mapped on top of standard RAM so that software does not need to access any kind of priviliged memory area.

JT: This means that customers don't have to worry about adding hooks in on either the operating system or application layer to get data out. Simics adds this support on the simulated hardware layer and thus makes execution easier and faster than when using real hardware.

JE: Thanks, I never knew that it was that difficult to get data out of a piece of hardware.

JT: This is something that varies greatly. For instance, if you have a JTAG programmer it is typically not too hard - but you do need to wait for it to finish flashing the ROM and executing. This can take up to several minutes. In other cases, for instance if all you have is a serial port load interface it can take longer. Often when the target environment goes from the development area to the test area, many of the extraneous interfaces are removed. So without the appropriate interfaces it can be very difficult and in certification environments having limited IO resources is very common.

As a result, if you have a target with some IO limitations, Simics can act as a replacement for test case execution. Especially in cases where it takes a long time to program the target FLASH and where IO is limited, Simics can be used as a drop-in component to speed the test execution process, both for system and unit test. Thus Simics can help our customers finish their certification and get their product to market faster.

JE: So how does the connection we have in place work in practice for a user of the LDRA TBrun tool ?

JT: In practice, the user selects 'Run Driver' which then, as is normally done, automatically invokes the compiler and instead of controlling a JTAG programmer or other device, LDRA Testbed and TBrun execute Simics to run through different test cases for unit and system test. When the execution is complete, Simics exits and passes the results back on the host file system where LDRA Testbed and TBrun expect it.

The image below shows LDRA TBrun executing a simulation on a Simics virtual machine and obtaining dynamic coverage.

Final notes: For more information about Simics, please contact your local Wind River sales representative.  For more information about LDRA’s tools or the integration they have with Simics, contact your local LDRA sales representative.