ISO/IEC JTC1/SC22/WG15 N391 July 1, 1993 ISO/IEC JTC1/SC22/WG15 Consultitive DEFAULT Letter Ballot: Proposed Disposition of Comments on DIS 9945-2 Please return by August 1, 1993 to: Jim Isaak Digital Equipment, ZKO2-3/R56 110 Spitbrook Rd. Nashua, NH 03062 (or FAX: +603 881 2790) Also, please DO email any comments to both Jim Isaak and Hal Jespersen so we can make any changes to the Disposition of Comments as needed [hlj@posix.com, isaak@decvax.dec.com] Please ONLY respond if you are the designated person in your country to respond for JTC1/SC22/WG15 with your national position on this. SUBJECT: The attached Proposed Disposition of Comments for DIS 9945-2 will be forwarded to the SC22 Secretariat as the WG15 recommendation. A lack of response to this ballot will be considered "approval" to forward the Disposition of Comments. Constructive comments will be included in the Disposition. Please surface substantive objections to forwarding this document quickly, and to all of WG15 via email. If 3 or more participants in WG15 express the need for further resolution in WG15 before we forward this to SC22, I will ask Joe Cote for an extension so we can obtain a better understanding and agreement. Please indicate your National Body: __________________________ ___ Approve (with comment ___) ___ Disapprove (Please comment) ___ Abstain Your name:____________________________________ Comments: July 1, 1993 [DRAFT] DISPOSITION OF COMMENTS FOR SC22 N1393 [Voting Summary for ISO/IEC DIS 9945-2: 1992] Hal Jespersen Project Editor, JTC 1/SC22/WG15 ISO/IEC DIS 9945-2: 1992 was approved by JTC 1 as an International Standard in voting that ended in June 1993, as described in SC22 N1393, "Summary of Voting and Comments on ISO/IEC DIS 9945-2 ...." This Disposition of Comments document describes how each of the comments received during the voting will be represented in the final IS document to be sent to ITTF in August 1993, or what further action to address these comments is proposed for the responsible working group. The dispositions were prepared by the WG15 Project Editor, Hal Jespersen, in consultation with the US development body [IEEE P1003.2 working group at their July meeting] and the member bodies of WG15 [electronic mail discussions, in accordance with our agreement in the May 1993 WG15 meeting]. BACKGROUND AND OPTIONS Since the 9945 standards are developed using the SC22 procedures for host-body development, and are coordinated between the IEEE and WG15 as governed by the SC22 Synchronization Plan [SC22 N1360], the disposition of comments on the DIS ballot should maintain synchronization between IEEE Std 1003.2-1992 and the International Standard. In keeping with this goal, there were four broad choices available to WG15: 1. Submit IEEE Std 1003.2-1992 as the IS. A few insignificant editorial changes (page headers, front matter, etc.) would be required and the IEEE would simply reprint the new version, but continue to label it as the 1992 IEEE standard. A disadvantage of this approach is that it would appear to ignore all DIS comments. 2. Submit IEEE Std 1003.2-1992 as the IS, as in choice 1, but make editorial changes sufficient to dispose of a number of the DIS comments. Unfortunately, the types of changes judged to be editorial by the IEEE are severely limited when compared to the editorial discretion normally afforded to an ISO project editor. Grammar and presentation can change, but no normative changes are allowed. Even changes to substantive informative annexes (such as rationale) are restricted. The project editor did receive permission from the IEEE to modify Annexes G and H to satisfy DIS comments since these informative annexes were essentially supplied by Denmark and WG15 and passed through without serious consideration by the IEEE balloting groups. 3. Correct a number of minor technical errors identified in the DIS comments, bypassing large or controversial changes. This would require that a new IEEE standard be approved, which consists of assembling a new balloting group, conducting a focused ballot limited to these changes, receiving IEEE Standards Board approval, and republishing the revised standard. 4. Address all comments from the DIS ballot and those that have been accumulated from previous ballots (such as the Canadian requirements for the pax utility format and Japanese character set and collation issues). This would also require a new IEEE standard, but, due to the wide range of changes in these comments, it would not be possible to closely focus the US balloting process to just those specific issues. Furthermore, since substantive changes would be made, a second, 3-month DIS ballot would have to be conducted. In their October 1992 meeting, the member bodies of WG15 gave clear guidance on the development path for 9945-2. They requested that the DIS be approved as an IS as soon as possible and that open technical issues be deferred to the first amendment, which also should be approved as soon as possible. Therefore, choice 2 was selected as the appropriate course. This choice approves the IS immediately, accounts for all open issues in changed informative annexes, and requires no IEEE balloting delays. The US development body should adopt all the open issues for its P1003.2b work and bring that work forward as a CD Amendment to 9945-2 as soon as possible. DISPOSITION OF COMMENTS The comments will result in four categories of changes from the DIS to ISO/IEC 9945-2: 1993. Many are represented in the revised text of Annex H, which is included as an attachment to this Disposition of Comments document. 1. Editorial. The following comments are editorial changes that will be included in the IS: Denmark.3, Japan.1, Japan.5, Japan.12. Note that requirements for specific formatting and document organization are governed by standing agreements between the IEEE and ITTF. Denmark.2: This proposed change to 18-point headings will be referred to the ITTF for consideration, but no delay in publication is planned to gain acceptance for this if there is any question. UK.1: This proposed change to include line numbers will be referred to the ITTF for consideration, but no delay in publication is planned to gain acceptance for this if there is any question. 2. Corrigenda. The following comments represent minor corrections and clarifications. They will be briefly listed in the revised Annex H with the expectation that they will be changed without controversy in the first amendment: Japan.6, Japan.7, Japan.8, Japan.9, Japan.10, Japan.11, Japan.13, Japan.15, Japan.16, Japan.21, Japan.24, Japan.27, Japan.29, Japan.31, Japan.32, Japan.33, Japan.35, Japan.36, Japan.44, Japan.45. 3. Investigation-1. The comments in this category require further investigation before being processed for the first amendment. They will be listed in Annex H as open issues to be evaluated and considered by the IEEE working group and WG15. The following comments are self- explanatory and are referred to SC22/WG15 for consideration as presented in the National Body Ballot item referenced here: Australia.2, Japan.14, Japan.17, Japan.20, Japan.22, Japan.23, Japan.25, Japan.26, Japan.28, Japan.30, Japan.34, Japan.43, Japan.46, Netherlands.1, Netherlands.2, Netherlands.3, Netherlands.4, Netherlands.5, Netherlands.7. 4. Investigation-2. The comments in this category require further investigation before being processed for the first amendment, as in item 3. However, they require additional explanation here of their disposition: Austria.1: A proposal of specific technical changes (or at least, requirements) to improve timer resolution is requested from Austria. Denmark.5: The requested change to the Danish example profile in Annex G will be made in the IS, but normative changes will be investigated for the amendment. Japan.2: Although this may seem editorial, there are difficulties associated with labeling utilities such as mailx where clear UPE subclause boundaries are not possible. Rather than changing only the easy subclauses, all related changes will be investigated for a consistent solution in the amendment. Japan.4, Japan.19, Japan.39, Japan.40: As implied by the comments, a proposal of specific technical changes is requested from Japan. Japan.41, Japan.42: A proposal of specific technical changes is requested from Japan. The changes should be consistent between these and the related group in Japan.4, above. Netherlands.6: This comment on stateful encoding conflicts with earlier requirements from Japan. The Netherlands are requested to participate in technical discussions on this subject between themselves, Japan, the US, and other interested parties. 5. Other. The following comments will result in no changes in the IS, for the reasons indicated: Australia.1: Although Australia offers no specific technical comments, it should be noted that the rationale to the standard does explain that the UPE commands are purposely less rigorous in terms of precise output descriptions because they are designed primarily for human users, who can easily react to minor differences in existing implementations, rather than shell scripts, which would have more difficulty doing so. Australia.3, Australia.4: No specific comments were offered on how some utilities can be over-specified, but with poor precision, or how they could be improved by reviews for consistency. More detailed technical proposals would be welcomed for the CD amendment. Canada.1: The value 2 for {POSIX2_COLL_WEIGHTS_MAX} in Table 2-17 does not mean that the maximum number of weights that can be accepted by the localedef utility is 2; it means that localedef must always support at least 2. (Note that the Table title is "Utility Limit Minimum Values".) It is true that no more than two weights should be specified for portable locale definitions. Note that the historical implementations of these utilities (before 9945-2 and IEEE Std 1003.2-1992) only provided a single weighting, and it seems that this has been sufficient for handling many natural language collation problems. There is no currently known minimum value that would satisfy all cultures, so any number selected greater than 2 would be guesswork and it would also impose burdens on implementations that serve cultures without greater requirements. Comparable restrictions on portable programs exist in POSIX.2. They cannot assume they have characters outside the ASCII set. The format of portable messages is a variant of English. Any extensions to these restrictions are handled outside 9945-2, by user requirements placed on vendors or by more formal profiles. Therefore, it would be appropriate for national bodies needing more weights to specify in a national profile that a larger minimum value be required for {COLL_WEIGHTS_MAX} in Table 2-18. Denmark.1, Denmark.4: The concept of "binary" or "compiled" locales has been quite popular among implementors of the standard and no attempt has been made to mandate interfaces that would make such implementations nonconforming. The "localedef copy" and "replace-after" modifications proposed here would make binary locales extremely difficult to support. Furthermore, they are merely alternatives to existing, standard UNIX text-file manipulation tools. Since these modifications have received little support in the WG15RIN after repeated discussions, and none from the US development body or any known implementors, they should not be required. Japan.3: The phrase "in the POSIX Locale" is never indicated to be equivalent to "in a locale upward-compatible from the POSIX Locale in some unspecified way," so any confusion in this area is unwarranted. The intent is that a portable application can always set LC_ALL=POSIX and receive the precise behavior guaranteed by the standard. Any other locale produces locale-dependent results. In the cases where the phrase "in the POSIX Locale" is used, it generally refers to the format of output messages, which may very likely change in other locales, even those that are quite similar to the POSIX Locale. Japan.18: The batch utility is fully specified by its equivalence to the specific command "at -q b -m now". In theory, all additional subclause headings could have been omitted and a 3 or 4-line utility description would have sufficed. However, the template of headings was retained for editorial consistency with the other utilities. Thus, the "See " entries are editorial courtesies to the reader and require no additional specification. Japan.37, Japan.38: As noted in objection Japan.38, there are two critical aims for the uuencode/uudecode utilities: (A) to convert a binary file file into a "printable" text data format, and (B) to create a portable interchange format of a binary file for delivery as text data through an appropriate transfer mechanism (such as the mailx utility). The reason for converting the data from the "ASCII" form into characters in the current locale is exactly to accomplish aim (B). This concern was raised during the IEEE balloting process. The changes proposed by Japan closely resemble the original text in early drafts of POSIX.2a. The arguments for using the current form specified in 9945-2 are in the rationale for the standard, but are restated here: 1. For locales using ISO 646, or ASCII or any underlying codeset using the characters with the same encoding as 646 for values 0x20 through 0x5f, no translation is needed. This behaves as everyone would expect even though a null translation is specified. 2. For locales using an underlying codeset with different encodings, such as EBCDIC, the ASCII encodings are translated to EBCDIC. This may seem intuitively wrong because the translated file now contains codeset-dependent characters in what was meant to be a portable file. However, the intended use of the uuencoded file must now be examined: E-Mail: The sole reason this utility was developed on UNIX systems was to allow a user to send e-mail containing binary data to another user, Therefore, the design of the encoding algorithm must be sensitive to the e-mail environment. Since e-mail networks (like the Internet) typically work with headers assuming ASCII, mail delivery agents running in locales that are not ASCII-compatible actually translate from the codeset of the source locale to ASCII (or 646 or 8859) before inserting text into the e-mail delivery system. Similarly, e-mail received from the network is translated into the underlying codeset of the receiving locale without the user taking any explicit action. Therefore, if uuencode did not translate to the current locale, the process of sending e-mail would take ASCII characters that were assumed to be EBCDIC and translate them into ASCII, producing nonsense. Tape/Disk: It would be possible (although not very useful) to uuencode a file, put it on a magnetic tape or diskette, take it to a system using a different underlying codeset for the C Locale, and try to uudecode the file. In this case, the POSIX.2 encoding algorithm would fail. However, this case is expected to be much less common than the e-mail case. The workaround is to use the iconv utility, if necessary, to create an ASCII-compatible version of the file to go on the tape/diskette and to use the iconv utility, if necessary, on the target machine to convert the ASCII file into the receiving locale's codeset. Note, however, that this same mechanism would need to be used for *any* text files being transferred by tape/diskette, as well as the uuencoded binary files. Therefore, these additional conversion steps should not be a problem. Since the algorithm in POSIX.2 correctly supports the intended usage of the utility (e-mail), no change in the algorithm is warranted. BALLOT COMMENT CROSS-REFERENCE Comment Disp Comment Disp Comment Disp ------- ---- ------- ---- ------- ---- Australia.1 5 Japan.12 1 Japan.34 3 Australia.2 3 Japan.13 2 Japan.35 2 Australia.3 5 Japan.14 3 Japan.36 2 Australia.4 5 Japan.15 2 Japan.37 5 Austria.1 4 Japan.16 2 Japan.38 5 Canada.1 5 Japan.17 3 Japan.39 4 Denmark.1 5 Japan.18 5 Japan.40 4 Denmark.2 1 Japan.19 4 Japan.41 4 Denmark.3 1 Japan.20 3 Japan.42 4 Denmark.4 5 Japan.21 2 Japan.43 3 Denmark.5 4 Japan.22 3 Japan.44 2 Japan.1 1 Japan.23 3 Japan.45 2 Japan.2 4 Japan.24 2 Japan.46 3 Japan.3 5 Japan.25 3 Netherlands.1 3 Japan.4 4 Japan.26 3 Netherlands.2 3 Japan.5 1 Japan.27 2 Netherlands.3 3 Japan.6 2 Japan.28 3 Netherlands.4 3 Japan.7 2 Japan.29 2 Netherlands.5 3 Japan.8 2 Japan.30 3 Netherlands.6 4 Japan.9 2 Japan.31 2 Netherlands.7 3 Japan.10 2 Japan.32 2 UK.1 1 Japan.11 2 Japan.33 2