From derek@knosof.demon.co.uk Wed Apr 12 15:20:20 1995 Received: from knosof.demon.co.uk by dkuug.dk with SMTP id AA27357 (5.65c8/IDA-1.4.4j for ); Wed, 12 Apr 1995 16:22:23 +0200 Received: from knosof.demon.co.uk by knosof.demon.co.uk with SMTP id AA810 ; Wed, 12 Apr 95 15:22:16 BST Date: Wed, 12 Apr 1995 15:20:20 GMT From: derek@knosof.demon.co.uk (Derek M Jones) Reply-To: derek@knosof.demon.co.uk Message-Id: <809@knosof.demon.co.uk> To: wg15@knosof.demon.co.uk Subject: Re: (wg15-uk 699) (SC22WG15.491) Feedback to WG15 on WG15 N511 from the U.S. WG15 TAG X-Mailer: PCElm 1.10 Lines: 81 X-Charset: ASCII X-Char-Esc: 29 All, In message <199504120605.AA15641@dkuug.dk> Stephen Walli writes: > WG15 TAG N503 > 18-Jan-1995 > Feedback to WG15 on WG15 N511 from the U.S. WG15 TAG > > This feedback responds to action item 9410-30 from the Whistler > WG15 meeting. The U.K. discussion paper (WG15 N511) presents concerns > that there needs to be some form of application conformance > testing and certification process to support the development > of ``POSIX'' applications, claiming that without this procedural support, > ``POSIX'' is doomed to failure. > > The U.S. WG15 TAG discussed WG15 N511, and believes that it understands > the reason for the request posed in WG15 N511, and has analyzed the request > along with the feasiblity and efficacy of such conformance testing. > We have the following concerns that we would like to contribute to the > discussion about application conformance testing: > > 2. POSIX.1 is not a complete solution. It builds on the > ISO C model of portability. It is not intended to be a complete > application source-code portability model by itself. > An interesting application may have a graphic user interface, > and implement an SGML browser or SQL based query engine. > The application's source-code may have been written to use curses, > Xlib, or some other API for its GUI. Just because POSIX.1 is unlikely to be a complete solution is not a reason for applications not be tested for conformance to it. The conformance statements contained within Posix (which are different from C and do not build on ISO C; Posix has a much more sophisticated model) allow for applications to use non-Posix API's. > 3. The inverse also exists; it is very possible for applications source-code > to be written to a POSIX.1/C Std library model of portability, that still > make all sorts of architecture dependent assumptions -- file layouts, > byte ordering -- that render the application unportable. We are not talking about portability we are talking about conformance to Posix, which is a step along the road to full blown portability. > > 4. Applications source code is developed to a POSIX.1-based model of > portability with the *intent* that it will be portable to other machines that > support the model. The only way of *knowing* that the application > does indeed work on the other machines is to build and test the application > on each of the other machines supporting the model. > > Technology exists today that statically scans source code to check it > against various different API models. Tools in this arena generally > support the functionality to modify the API models to be scanned. > None of the tools in the knowledge of the U.S. MB appears to completely > deal with dynamic behaviour, or the subtle unportable constructs that > may exist in an application's source code. No test suite can show that an application/implementation is fully conformant to a standard. The Posix.1 tests often only check some system calls with only one case. But do we say that implementation testing is a waste of time? Saying that Applications testing should not be done because it will never be 100% perfect is not a reason, it is an excuse. > > 5. Users purchasing applications should not care how the binary is created. > The buyer of an application binary cares that the application performs > its desired job, and is available and supported on the particular platforms > that the user has or intends to have. While a partial predicative indicator > of the portability of the source code from which this application binary > is derived is of some benefit to the application binary customer and > user, it is not sufficient to ensure POSIX.1 conformance. Not all users simply buy binaries. Some buy source. Even if they did buy some binaries there is the feel good factor of knowing that the vendor will have little trouble porting to new platforms (purchases are often made on the understanding that a product will be made available on other machines as time progresses). derek