From isaak@codjig.zko.dec.com Thu Mar 11 08:10:02 1993 Received: from crl.dec.com by dkuug.dk with SMTP id AA21801 (5.65c8/IDA-1.4.4j for ); Thu, 11 Mar 1993 19:06:53 +0100 Received: by crl.dec.com; id AA20519; Thu, 11 Mar 93 13:07:11 -0500 Received: by codjig.zko.dec.com (5.57/fma-100391/BobG-15-Feb-93);id AA12507; Thu, 11 Mar 93 13:10:02 -0500 Date: Thu, 11 Mar 93 13:10:02 -0500 From: isaak@codjig.zko.dec.com (Jim Isaak-respond via isaak@decvax.dec.com) Message-Id: <9303111810.AA12507@codjig.zko.dec.com> To: sc22wg15@dkuug.dk Subject: Conformance Testing Model NP proposal Cc: rmartin@swe.ncsl.nist.gov X-Charset: ASCII X-Char-Esc: 29 WG15 folks, I expect to circulate this in the WG15 mailing as an expert contribution, although I may hear from David Cannon that it is a formal UK contribution. In any case, it is of value to have insight on this proposal and to give it thought before our April meeting. (I would expect a recomendation from RGCT on this as input to WG15 as well.) =============================== Jim Isaak ========================================================== Proposed Draft NP March 3, 1993 from: + Colin O'Driscoll + + Open Systems Group + + National Computing Centre + + Oxford Road + + Manchester + + M1 7ED + + U.K. + + phone: +44 (0)61 228 6333 + + fax: +44 (0)61 236 4715 + + email: colin@ncc.co.uk, ...nccotd!colin + ___________________________________________________________________________ ---------------------------------------------------------------------- Title (subject to be covered and type of standard, e.g. terminology, method of test, performance requirements, etc.) Conformance Testing Reference Model for Open Systems Environments. ---------------------------------------------------------------------- Scope (and field of application) Those areas of standardisation that will be covered by TR 10000-3. ---------------------------------------------------------------------- Purpose and justification GENERAL "POSIX" testing methodology is less mature than its OSI equivalent. Current IEEE work on conformance testing has concentrated on the issue of test suite derivation from base standards (test assertions, the concept of test suite conformance). It is necessary to establish an overall model for OSE Conformance Testing which complements IEEE 1003.3-1991, and which is as far as possible compatible with existing OSE work. The conformance testing of OSE profiles containing both OSI and non-OSI components must be handled in as coherant and consistent a way as possible, in order that users have confidence in procuring against such composite ISPs. The production of an overall reference model will assist in this process. SPECIFIC AREAS FOR STANDARDISATION General Terminology One of the barriers to harmonisation of OSE Conformance Testing is the lack of a common vocabulary. The first task should be to establish common terminology in as many areas as possible and to clearly identify unavoidable areas of difference. The Conformance Assessment Process IEEE 1003.3-1991 contains no overall model of the Conformance Assessment Process. This is a quality hazard. Without an overall model it is not obvious that test results are repeatable or reproduceable. Therefore this standard will define an overall model of the Conformance Assessment Process. Validation and Maintenance of Test Tools. IEEE 1003.3-1991 lays down conformance requirements for test tools but does not say how conformance can be established and maintained. This is also a quality hazard. Therefore this standard will define methods for validation and manitenance of test tools. ---------------------------------------------------------------------- Programme of work a multi-part International Standard consisting of 3 parts, which will be Terminology, Conformance Assessment Model and Validation and Maintenance of Test Tools. ---------------------------------------------------------------------- Relevant documents to be considered The ISO 9646 series of standards, EN 45001 to EN 45014, IEEE 1000.3-1991 and TR 10000. ---------------------------------------------------------------------- Co-operation and liaison ISO/IEC JTC1 SGFS ISO/IEC JTC1/SC22/WG15 ISO/IEC JTC1/SC21/WG1 IEEE P2003 ---------------------------------------------------------------------- Preparatory work offered with target dates Working Draft Q2 1994 ---------------------------------------------------------------------- Will the services of a maintenance agency or registration authority be required? - No Are there any known requirements for coding? - No Does the proposed standard concern known patented items? - No ---------------------------------------------------------------------