Date: Thu, 16 Nov 95 13:26:58 GMT
From: jon@national-computing-centre.co.uk (Jon Leigh)
To: D.Cannon@exeter.ac.uk
Subject: Re: Guide to Conformance Testing Methodology - NP

Dear Dave

>
> Hi Jon,  one of the actions to come out of last months WG15 meeting
> was on the UK to decide whether it continued to be interested in the
> Comprehensive Test Methods NP which Colin O'Driscoll had an interest
> in eighteen months/two years back.
>
> Unfortunately the action on the original NP, which was for SC22 to
> circulate it to other potentially interested SCs to see if they wished
> to participate in the work, was lost down the cracks when the SC22
> secretariat moved.
>
> I've appended a copy of the NP below;  if we are still interested in
> getting this off the ground then WG15 asked if anything in it should be
> updated or changed, since we're a little older than we were when it was
> first created.
>
> Look forward to hearing from you,   cheers,
>

You will not be surprised to hear that unfortunately NCC can no longer
champion this new NP i.e. we have no resource to offer.

Having said that I also think that the work is now being championed by EWOS -
from the OSE perspective rather than simply POSIX.

As you know with the (now official) demise of the RGCT we latterly turned our
attentions to EWOS who through the EG/CT and EG/OSE groups have been working
on a document on OSE Profile Conformance Testing.

The document they have been developing is now near to completion - the latest
version I have is entitled 'EWOS Technical Guide on OSE Profile Conformance
Testing' (EWOS/EG-CT/95/20 Issue 4.0) issued on the 4 October. This document
has been developed to complement an EWOS document on the development and use
of OSE profiles (EWOS/EG/OSE/95/42). That document recognises that OSE profiles
will involve more than one type of interface (e.g Application Program
Interfaces, Communication Service Interfaces etc..). It is therefore recognised
that to test OSE profiles will require the development of a new OSE Profile
Conformance Testing Methodology, which should be based on combining and
expanding existing methodologies. Two of the primary existing methodologies
used as input to this document are POSIX (for API's) and OSI (for CSI's).

The EWOS work is particularly relevant since the latest update to the  P2003
(D6.0) document 'Requirements and Guidelines for Test Method Specifications
for Measuring Conformance to PASC Standards' also to some extent looks at the
profiling (just from the API perspective) and has references to ISO 9646.
EWOS has also developed a Conformance Testing Vocabulary document
(EWOS/EG/CT/94/182) which includes both POSIX and OSI terminology (this
therefore covers part of the NP).

It therefore seems sensible to simply refer WG15 to this work and drop the NP.

> PS:  I think I remember you saying that you had received a copy of
> P2003.2 D10 - the 1500 page doorstop - a while back.	Have you been able
> to look at it at all?  Any comments you have would be welcome!   Dave
>

"It is enormous!!"

To be honest although we are starting to actively use it with respect to our
POSIX.2 Testing Service as a reference point for analysis of test results we do
not intend to read it cover to cover (surprise, surprise).

One of my staff (Gill Howard) did read the first 150 pages before begging for
mercy.

The only issue she found seems to be related to how assertions are derived
from the base standard:

	In section 3.9.4.4 of the base standard the 'if Conditional Construct'
	indicates in the format that the 'if' construct is terminated with the
	reserved word 'fi' - but since this is not stated in the accompanying
	text then no assertion exists to test it in the Test Methods standard.

That apart we have no comment and are not likely to have - it is really only the
test suite developers (X/Open) who can comment since they have actually used
the standard (though VSC4 is written against a number of different drafts of
the Test Methods standard).

Sorry I cannot offer more.

I will hopefully be able to make the POSIX Panel meeting - so I will see you
there.

Best Wishes

Jon