What is the difference between compliance and conformance
and what does that mean to you and me? Or more importantly what does this mean
to your project?
Well, compliance is an informal industry
term generally accepted to mean the system provides support for some of a given standard. Technically, an OS that provides only one API of a POSIX standard is "compliant," though not very useful. Vendors of compliant systems are generally expected to offer documentation describing which parts of the standard are supported, and which are not.
Wind River for instance, has such a statement to show "compliance" levels
of its VxWorks Operating System against the various POSIX standards. This reads like 100%
conformance to SCA 2.2.1 (another POSIX Subset for Software Defined Radio)
Conformance on the other hand is a recognition of formal testing, that prove that an
operating system provides 100% support for a given standard. Certification of conformance is awarded to an operating system after an official Certification Authority has reviewed not only the results of formal testing, but formal conformance documentation as well. In the case of POSIX this is well covered on the
IEEE and The Open Group sites. Products that are POSIX certified can claim conformance, and in doing so also publish a list of which of the optional components they support to exactly define what support they provide.
This statement also lists the configuration of the system, and the test environment required to re-produce the tests.
To do this the company involved must purchase an official test suite from a certification authority , run the tests and submit the results for certification for conformance. As of November 2003, The IEEE and The Open Group extended POSIX Certification to the 2003 Edition of IEEE Std 1003.1. This has subsequently been extended to include IEEE Std 1003.13-2003 (what we are interested in), for both the PSE54 Multipurpose Profile and the PSE52 Real time Controller Profile. The test results are supplied to the Certification Authority such as the IEEE or The Open Group, in order to audit the results and confirm they have all passed. This along with the Conformance report then give a company the right to say they are conformant to the standard tested against.
Certified conformance is obviously a much better indication as to a product ‘ s suitability, if POSIX is a requirement for your system, as you don’t have to rely on marketing data sheets showing "compliance" but know the product has been audited and tested to the standard involved. You as an application builder can then tick the box and get on with developing a product just using the chosen standard.
As John Jones pointed out on my last blog only one company is listed as having run these tests and that for 1003.1, the complete POSIX standard, this in itself is interesting as the older POSIX standard 1003.1-1998 had many companies listed against it (from memory as I could not find the old listing anymore!
I wonder why this is?
One reason for the lack of testing at the Real Time OS level is that the PSE52 and PSE54 tests are relatively new, not only to the POSIX community, but also to the DSO world and you have to adapt these tests to run in a DSO environment (such as porting the test suite from a UNIX based self-hosted environment to run in a client-server mode, as a device target doesn’t always have the system resources to support running the large test framework)
Another reason could be the usual product development issue of scheduling in these additional workloads into an already heavily oversubscribed feature development schedule along with demands for more "exiting" technologies such as Mobile-IP [Webopedia] or Remote Diagnostics [Wind River Management Suite].
Or maybe the demand to see these POSIX certified products just is not there yet?