To: sc22wg15@dkuug.dk
Subject: Feedback to WG15 on WG15 N511 from the U.S. WG15 TAG
Date: Tue, 11 Apr 1995 22:05:54 -0400
From: stephe@srw.com (Stephen Walli)

                                                           WG15 TAG N503
                                                             18-Jan-1995
Feedback to WG15 on WG15 N511 from the U.S. WG15 TAG

This feedback responds to action item 9410-30 from the Whistler
WG15 meeting.  The U.K. discussion paper (WG15 N511) presents concerns
that there needs to be some form of application conformance
testing and certification process to support the development
of ``POSIX'' applications, claiming that without this procedural support,
``POSIX'' is doomed to failure.

The U.S. WG15 TAG discussed WG15 N511, and believes that it understands
the reason for the request posed in WG15 N511, and has analyzed the request
along with the feasiblity and efficacy of such conformance testing.
We have the following concerns that we would like to contribute to the
discussion about application conformance testing:

1. POSIX.1 (ISO/IEC 9945-1:1990) and its proposed amendments were
indeed developed to ``support applications portability at the source-
code level.''  The rest of the first paragraph identifies the scope
and audience of the material in the standard as application developers
and system implementors.  This is not to say that another constituency
of users of the standard may not make use of material in the standard,
but they run the risk of using the tool for a purpose for which it is
not suited.

Users that buy ``POSIX'' systems, or more accurately,
systems that have been certified in some manner (either via
X/Open or NIST) as ISO/IEC 9945-1:1990 conformant should
be purchasing these systems because there exists applications source
code written to use functionality defined in the POSIX.1 specifications
that will be built into executables to run on these platforms.

2. POSIX.1 is not a complete solution.  It builds on the
ISO C model of portability.   It is not intended to be a complete
application source-code portability model by itself.
An interesting application may have a graphic user interface,
and implement an SGML browser or SQL based query engine.
The application's source-code may have been written to use curses,
Xlib, or some other API for its GUI.

Interesting applications likely make use of more than simply POSIX.1 and C.

3. The inverse also exists; it is very possible for applications source-code
to be written to a POSIX.1/C Std library model of portability, that still
make all sorts of architecture dependent assumptions -- file layouts,
byte ordering -- that render the application unportable.

4. Applications source code is developed to a POSIX.1-based model of
portability with the *intent* that it will be portable to other machines that
support the model.  The only way of *knowing* that the application
does indeed work on the other machines is to build and test the application
on each of the other machines supporting the model.

Technology exists today that statically scans source code to check it
against various different API models.  Tools in this arena generally
support the functionality to modify the API models to be scanned.
None of the tools in the knowledge of the U.S. MB appears to completely
deal with dynamic behaviour, or the subtle unportable constructs that
may exist in an application's source code.

These scanning tools are powerful additions to a programmer's
tool chest to aid in the development of more portable source code.
-- They can catch obvious mis-use of APIs.
-- They can act as a predictive tools that give the programmer
   some confirmation that if the source-code ``passes'' the analysis,
   the source code *should* build on a system supporting the
   same API model.
-- A customer of source code has an ability to perform rough analysis of
   the deliverable.  (It is likely that there are other issues beyond
   predictive portability that are of interest to a source code
   customer.)

While this tool support is almost essential to an application programmer
developing portable source code, it is not sufficiently exact at this
time to base application branding or certification programs on.
This tool support has nothing to do with POSIX.1 support in and of itself.

5. Users purchasing applications should not care how the binary is created.
The buyer of an application binary cares that the application performs
its desired job, and is available and supported on the particular platforms
that the user has or intends to have.  While a partial predicative indicator
of the portability of the source code from which this application binary
is derived is of some benefit to the application binary customer and
user, it is not sufficient to ensure POSIX.1 conformance.

An example of this is a database engine.  The application users may
develop the code using POSIX.1 to perform all process management,
and exploit knowledge of each architecture upon which the binaries ship
to ensure that queries are as fast as possible.  On some architectures
this may be the POSIX.1 I/O calls, and on other architectures, something
entirely different.  The user should care that the database application is as
fast as possible, not what the application source-code writer used to
create it.

This also creates the situation that the ``application''
is ``POSIX.1 conformant'' on one system and not on another.