Internet Draft
Internet-Draft E. Wada
Fujitsu Laboratories
Expires in six months J. Murai
Keio University
K. Yamamoto
IIJ Research Laboratory
January, 1999
Japanese Character Encoding Scheme for Internet Messages
<draft-yamamoto-charset-iso-2022-jp-02.txt>
Status of this Memo
This document is an Internet-Draft. Internet-Drafts are working
documents of the Internet Engineering Task Force (IETF), its areas,
and its working groups. Note that other groups may also distribute
documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other documents
at any time. It is inappropriate to use Internet-Drafts as
reference material or to cite them other than as "work in progress".
To view the entire list of current Internet-Drafts, please check the
"1id-abstracts.txt" listing contained in the Internet-Drafts Shadow
Directories on ftp.is.co.za (Africa), ftp.nordu.net (Northern
Europe), ftp.nis.garr.it (Southern Europe), munnari.oz.au (Pacific
Rim), ftp.ietf.org (US East Coast), or ftp.isi.edu (US West Coast).
Abstract
This memo describes the character encoding scheme used in electronic
mail, NetNews, and world wide web messages in Japanese networks. It
was first specified by and used in JUNET then described in RFC 1468.
The name of this character encoding scheme was originally known as
'JUNET code' and is now called 'ISO-2022-JP' when used in the
context of MIME.
In ISO-2022-JP text, both one 7-bit byte Latin script (ASCII or JIS
X 0201 Latin set) and two 7-bit bytes Kanji, Hiragana, Katakana and
some other symbols and characters (JIS X 0208) are employed.
Switching the graphic character sets is based on the extension
techniques defined in ISO 2022.
This memo eliminates some ambiguities of RFC 1468. However, it
NEVER introduces any essential changes against RFC 1468. Since the
character encoding scheme is now widely used in Japanese IP
communities, backward compatibility is most important in this
revision.
This memo revises RFC 1468 on the following points:
Yamamoto [Page 1]
Internet-Draft ISO-2022-JP CHARSET January 1999
1. It is clarified that ISO-2022-JP does NOT conform to ISO 2022.
2. The formal syntax is divided into two new yet compatible rules,
namely, ISO-2022-JP decoding syntax and ISO-2022-JP encoding
syntax.
3. The bit combinations permitted in JIS X 0208 are explicitly
described in the syntax so that the invalid character positions
will be excluded in ISO-2022-JP text.
4. Recommended graphic character sets are specified. That is, ASCII
is RECOMMENDED rather than JIS X 0201 1976 Latin set. ONLY to
embed YEN SIGN and OVER LINE, JIS 0201 Latin set MAY be used.
Also, JIS X 0208 1983 (including 1990) is RECOMMENDED rather
than JIS X 0208 1978.
1. Introduction
Efforts to exchange Japanese text via electronic mail[MAIL] and
NetNews[NETNEWS] messages started in the early days of JUNET[JUNET],
consisted of networks arbitrarily connected by UUCP in Japan. There
was strong demand for exchanging ASCII[ASCII], JIS X 0201 Latin
set[JISX0201-76], and JIS X 0208 graphic character sets.
JIS X 0201 Latin set is identical to ASCII except for REVERSE
SOLIDUS (BACKSLASH) and TILDE. REVERSE SOLIDUS and TILDE are
replaced by YEN SIGN and OVER LINE, respectively. This set is
Japan's national variant of ISO 646[ISO646]. Note that section
4.1.2 of RFC 2046 discourages use of ISO 646 in electronic mail.
The JIS X 0208 graphic character sets consist of Kanji, Hiragana,
Katakana and some other symbols and characters. Each character
takes up two bytes.
At that time, some implementations of message transfer agents (MTA)
had 7-bit restriction in their transport connection. Moreover,
European countries used 8-bit text to convey their languages. It
was thus crucial to design a 7-bit safe format to carry Japanese
text both for robustness against such MTAs and for distinction from
European languages.
Popular character encoding schemes(CES) for Japanese graphic
character sets were EUC-JP and Shift_JIS. However, they were not
chosen for a transfer form of Japanese text because they are 8-bit.
To encode ASCII, JIS X 0201, JIS X 0208 1978[JISX0208-78] and
1983[JISX0208-83], JUNET code was designed. This CES switches these
four graphic character sets according to the extension techniques
defined in ISO 2022[ISO2022], which is described later in this memo.
During JUNET days, some systems erroneously used the escape sequence
"ESC ( H" for JIS X 0201 1976 due to a typological error in the JIS
book. This escape sequence is officially registered for a Swedish
graphic character set[ISOREG] and MUST NOT have been used in JUNET
code. JUNET had difficulty in eliminating this misusage.
In 1993, the CES was described in RFC 1468[OLDSPEC] and the name
"ISO-2022-JP" was given to identify it in the context of MIME[MIME].
Yamamoto [Page 2]
Internet-Draft ISO-2022-JP CHARSET January 1999
The word "encoding" is ambiguous because it is used for both the
extension techniques and for conversion of MIME's transport safe.
For this reason, throughout this memo, the word "encoding" is always
preceded by a prefix word to clarify its semantics and the word
"charset"[CHARSET] is introduced to describe ISO-2022-JP CES.
"ISO-2022-JP encoding" means creation of ISO-2022-JP text by MIME
composers while "ISO-2022-JP decoding" refers to interpretation of
ISO-2022-JP text by MIME viewers.
At the time of writing RFC 1468, one more revision for JIS X 0208,
namely 1990[JISX0208-90], was available. RFC 1468, however, does
NOT suggest using the revision identification sequence "ESC & @"
even if two characters added in JIS X 0208 1990 are used.
This choice was practical from the empirical point of view but it
was a departure from the ISO 2022 platform. Strictly speaking,
ISO-2022-JP does NOT conform to ISO 2022 though the name is
associated with ISO 2022.
The purposes of this memo are mainly to clarify ISO-2022-JP CES and
to fix a bug of ABNF[ABNF] in RFC 1468, which failed to express null
lines. To avoid confusion, NO new escape sequence for JIS X 0208
1990 NOR for JIS X 0212 1990[JISX0212-90] is introduced. Such
efforts should be elaborated in other rich charsets.
NOTE: One more revision for JIS X 0208, namely 1997, is available
[JISX0208-97]. Since it is not a new graphic character set, most
parts of this memo do NOT explicitly talk about it. Likewise, JIS X
0201 1997[JISX0201-97] does NOT appear in the main body of this
memo. For more information, see Historical Note.
2. Standard Keywords
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [KEYWORDS].
3. Description
For historical reasons, ISO-2022-JP can contain four graphic
character sets, ASCII, JIS X 0201 1976 Latin set, JIS X 0208 1978
and JIS X 0208 1983 (including 1990). A charset should start with
and should end in ASCII since many message systems assume so.
Moreover, each line should be self-composite so that lines can be
safely split. ISO-2022-JP was designed to satisfy these
requirements. Its definition is as follows:
ISO-2022-JP text MUST start with ASCII and MUST end with ASCII.
Each line of ISO-2022-JP text MUST end with ASCII or JIS X 0201 1976
(i.e. before CRLF). The next line starts in the graphic character
set that was switched to before the end of the previous line.
ISO-2022-JP is NOT completely self-composite because a starting
Yamamoto [Page 3]
Internet-Draft ISO-2022-JP CHARSET January 1999
graphic character set of a line is not known if the previous line is
missing. This is one of the reasons to recommend ASCII, instead of
JIS X 0201 1976, in ISO-2022-JP.
An escape sequence is used to specify a start boundary of a graphic
character set different form the preceding graphic character set.
The following table gives the escape sequences and the graphic
character sets used in ISO-2022-JP text. The "ISOREG" is the
registration number in ISO's registry[ISOREG].
ESC Seq Graphic Character Set ISOREG
ESC ( B ASCII 6
ESC ( J JIS X 0201 1976 (Latin set) 14
ESC $ @ JIS X 0208 1978 42
ESC $ B JIS X 0208 1983 87
For example, the escape sequence "ESC $ B" indicates that the bytes
following this escape sequence are some characters of JIS X 0208
1983. To switch back to ASCII, the escape sequence "ESC ( B" is
used.
For the control characters in ASCII, [MIME] and [MAIL2] define CR
and LF only. Section 4.1.2 of RFC 2046 says that HT and FF have de
facto meaning. To make ISO-2022-JP syntactically and semantically
larger than US-ASCII, this memo defines the semantics of ESC only.
Semantics of other control characters in ASCII is outside the scope.
Note that the extension techniques defined here switches graphical
character sets, not control character sets. So, ISO-2022-JP CES
assumes that %d0-31 is always the control character set in ASCII.
If Japanese text represented within the four graphic character sets
is transferred by SMTP[SMTP] or NNTP[NNTP], it is STRONGLY
RECOMMENDED to be in ISO-2022-JP form.
Not all implementations distinguish JIS X 0208 1978 and 1983.
Moreover, if ISO-2022-JP text including both ASCII and JIS X 0201
1976 AND/OR ISO-2022-JP text including both JIS X 0208 1978 and JIS
X 0208 1983 is converted to EUC-JP or Shift_JIS, original
ISO-2022-JP text could not be re-produced.
So, to implement best interoperability, message composers SHOULD use
ASCII and JIS X 0208 1983 ONLY. However, message composers MAY use
JIS X 0201 1976 to embed YEN SIGN and OVER LINE. Message viewers
MUST accept all four graphic character sets. Please note that the
best strategy for interoperability is "TO BE LIBERAL IN WHAT YOU
ACCEPT, AND CONSERVATIVE IN WHAT YOU SEND"[GUIDE].
Yamamoto [Page 4]
Internet-Draft ISO-2022-JP CHARSET January 1999
JIS X 0201 Kana set MUST NOT be used in ISO-2022-JP text. It may
erroneously appear in ISO-2022-JP text with the following sequences:
<8-bit Kana set char>
SO <7-bit Kana set char> SI
SO <8-bit Kana set char> SI
ESC ( I <7-bit Kana set char> ESC ( B
ESC ) I <8-bit Kana set char>
JIS X 0212 1990 MUST NOT be embedded in ISO-2022-JP text. It may
erroneously appear following "ESC $ ( D" in ISO-2022-JP text. To
embed JIS X 0212 1990, other charsets such as ISO-2022-JP-2
[ISO-2022-JP-2] SHOULD be used.
IMPLEMENTATION NOTE: Even if a message composer does not provide any
input method for graphic character sets which is not used in
ISO-2022-JP, such as JIS X 0201 Kana set and JIS X 0212 1990, there
are ways to include them by accident. Typical cases are citing and
forwarding a message including JIS X 0201 Kana set and/or JIS X 0212
1990 (but labeled as ISO-2022-JP).
IMPLEMENTATION NOTE ON CITATION: If JIS X 0201 Kana set is cited, it
MUST be removed anyway. One possible deletion is to convert the
characters to corresponding characters in JIS X 0208 1983. Please
note that as of this writing there doesn't exist a charset to
contain JIS X 0201 Kana set. When JIS X 0212 1990 is cited, MIME
composers MUST take one of the followings: (1) Remove all characters
in JIS X 0212 1990 and specify ISO-2022-JP for the charset
parameter. (2) Specify other richer charset such as ISO-2022-JP-2.
IMPLEMENTATION NOTE ON FORWARDING: Since this memo talks about
ISO-2022-JP text only, forwarding an entire message is outside the
scope. But MIME composers are REQUIRED to forward valid messages in
the context of MIME. Mismatch between the charset parameter and its
content body MUST be eliminated. The situation would be mixed up
when the invalid message was digitally signed. But again,
procedures to create valid messages are outside the scope of this
memo.
4. Formal Syntax
Two sets of formal syntax rules are defined in this memo. One is
ISO-2022-JP decoding syntax for message viewers and the other is
ISO-2022-JP encoding syntax for message composers. ISO-2022-JP
decoding syntax is designed to be redundant to maintain backward
compatibility with existing message composers while ISO-2022-JP
encoding syntax is designed to be necessary and sufficient. Readers
are assumed to be familiar with the notational conventions defined
in [ABNF] and [MAIL2].
Yamamoto [Page 5]
Internet-Draft ISO-2022-JP CHARSET January 1999
4.1 Rules for ISO-2022-JP decoding syntax
ISO-2022-JP decoding syntax that embeds ASCII, JIS X 0201 1976, JIS
X 0208 1978, JIS X 0208 1983 (including 1990) is defined as follows
(see also Figure 1):
iso-2022-jp-text-d = *( line-d CRLF ) line-d
; When text is terminated by CRLF, the last 'line-d' is empty.
line-d = *single-byte-char-d
*(*double-byte-segment-d single-byte-segment-d)
; line-d MUST be limited to 998 bytes according to [MAIL2].
single-byte-segment-d = single-byte-designator-d *single-byte-char-d
single-byte-designator-d = %d27 %d40 ( %d66 / %d74 )
; ESC "(" ( "B" / "J" )
single-byte-char-d = %d0-9 / %d11-12 / %d14-26 / %d28-127
; All bit combinations of ASCII code excluding
; LF(%d10), CR(%d13), and ESC(%d27).
double-byte-segment-d = double-byte-designator-d *double-byte-char-d
double-byte-designator-d = %d27 %d36 ( %d64 / %d66 )
; ESC "$" ( "@" / "B" )
double-byte-char-d = %d33-126.33-126
; All bit combinations of 94 x 94 graphic character set
NOTE: ISO-2022-JP decoding syntax above describes valid byte
combinations for ISO-2022-JP. Other byte patters are invalid.
Message viewers MUST handle any combinations of %d0-255 and MUST
safely ignore the invalid patterns.
NOTE: The old specs, RFC 822 and RFC 1468, allow bare CR and bare
LF. MIME prohibits them. Though [MAIL2] allows message viewers to
accept bare CR and bare LF, it prohibits message composers from
generating them. This memo conforms this recent trend. Bare CR and
bare LF are irrelevantly accepted according to the note above. The
ISO-2022-JP encoding syntax defined below prohibits message
composers from generating them.
Yamamoto [Page 6]
Internet-Draft ISO-2022-JP CHARSET January 1999
+---------------------->
| +----------> |
| +--+ | +--+ | |
+----------> | +->|1S|-> +->|1C|->->-> exit
| +--+ | | | +--+ | +--+ | |
line-d --> +->|1C|->-> | <-------+ v
| +--+ | | +----------> |
<-------+ | +--+ | +--+ | |
+->|2S|-> +->|2C|->->
| +--+ | +--+ | |
| <-------+ |
<--------------------+
Figure 1: 'line-d' for ISO-2022-JP decoding syntax
NOTE: 1S, 1C, 2S, and 2C stand for 'single-byte-designator-d',
'single-byte-char-d', 'double-byte-designator-d', and
'double-byte-char-d', respectively.
4.2 Rules for ISO-2022-JP encoding syntax
To create ISO-2022-JP text or string, only characters in ASCII and
JIS X 0208 1983 (including 1990) SHOULD be used. But if YEN SIGN
and/or OVER LINE of JIS X 0201 1976 are to be used, the following
rules SHOULD be followed:
(1) It is RECOMMENTED to use YEN SIGN(%d33.111) and OVER
LINE(%d33.49) of JIS X 0208 1983 instead of YEN SIGN(%d92) and
OVER LINE(%d126) of JIS X 0201 1976, respectively.
(2) If sequences of JIS X 0201 1976 Latin set can be produced,
YEN SIGN and/or OVER LINE of the set MAY be used preceded by the
designator.
(3) Otherwise, YEN SIGN and OVER LINE of JIS X 0201 1976 SHOULD
be converted to REVERSE SOLIDUS and TILDE of ASCII,
respectively.
IMPLEMENTATION NOTE: Display of YEN SIGN and OVER LINE of JIS X 0201
1976 is highly dependent on the receiving system. REVERSE SOLIDUS
and TILDE of ASCII may be displayed. And again, if ISO-2022-JP text
including both ASCII and JIS X 0201 1976 is converted to EUC-JP or
Shift_JIS, original ISO-2022-JP text could not be re-produced. This
would cause verification failure of digital signature.
The following is ISO-2022-JP encoding syntax to accomplish rule (1)
or (3) above (see also Figure 2)
iso-2022-jp-text-e = *( line-e CRLF ) line-e
; When text is terminated by CRLF, the last 'line-e' is empty.
line-e = *single-byte-char-e [*i-segment-e f-segment-e]
; line-e MUST be limited to 998 bytes and SHOULD be limited to
; 78 bytes according to [MAIL2].
Yamamoto [Page 7]
Internet-Draft ISO-2022-JP CHARSET January 1999
i-segment-e = double-byte-designator-e 1*double-byte-char-e
single-byte-designator-e 1*single-byte-char-e
f-segment-e = double-byte-designator-e 1*double-byte-char-e
single-byte-designator-e *single-byte-char-e
single-byte-designator-e = %d27 %d40 %d66
; ESC "(" "B"
single-byte-char-e = %d1-9 / %d11-12 / %d14-26 / %d28-127
; All bit combinations of ASCII code excluding
; NUL(%d0), LF(%d10), CR(%d13), and ESC(%d27).
double-byte-designator-e = %d27 %d36 %d66
; ESC "$" "B"
double-byte-char-e = %d33.33-126 /
%d34.(33-46 / 58-65 / 74-80 / 92-106 / 114-121 / 126) /
%d35.(48-57 / 65-90 / 97-122) /
%d36.33-115 /
%d37.33-118 /
%d38.(33-56 / 65-88) /
%d39.(33-65 / 81-113) /
%d40.33-64 /
%d48-78.33-126 /
%d79.33-83 /
%d80-115.33-126 /
%d116.33-38
; The valid bit combinations of JIS X 0208 1990.
+-------------------------------------->
| +----------> |
| | +--+ | |
+----------> | | +->|1C|->->-> exit
| +--+ | | +--+ +--+ +--+ | | +--+ |
line-e --> +->|1C|->->->->|2S|->->|2C|->->|1S|-> <-------+
| +--+ | | +--+ | +--+ | +--+ | +--+
<-------+ | <-------+ +-->->|1C|->->
| | +--+ | |
| <-------+ |
<------------------------------------+
Figure 2: 'line-e' for ISO-2022-JP encoding syntax
NOTE: 1S, 1C, 2S, and 2C stand for 'single-byte-designator-e',
'single-byte-char-e', 'double-byte-designator-e', and
'double-byte-char-e', respectively.
5. MIME Considerations
This section discusses how to use ISO-2022-JP text or string
(i.e. excluding CRLF) in the context of MIME. The scope is limited
to the case where messages are transferred by SMTP[SMTP] or
Yamamoto [Page 8]
Internet-Draft ISO-2022-JP CHARSET January 1999
NNTP[NNTP]. Other protocols are outside the scope of this memo.
5.1 The Charset Parameter
To identify the charset described in this memo, "ISO-2022-JP" MUST
be used for the charset parameter wherever it is allowed. Please
note that the charset parameter is case-insensitive.
For the charset parameter, MIME composers SHOULD choose as small
charset as possible. For example, if a content body is ASCII,
US-ASCII SHOULD be specified instead of ISO-2022-JP. This rule
improves interoperability with other MIME viewers which do not
support various charsets.
To the contrary, MIME viewers SHOULD be able to handle text labeled
ISO-2022-JP even if its content is ASCII only. Remember the
liberal/conservative rule.
5.2 Content Transfer Encoding
ISO-2022-JP text is already in 7-bit form and MIME-encoding, such as
Base64 and Quoted-Printable, makes ISO-2022-JP text unreadable for
non-MIME viewers. For this reason, MIME composers SHOULD choose
"7bit" (case-insensitive) and SHOULD NOT MIME-encode ISO-2022-JP
text with Base64 nor Quoted-Printable.
Note that "7bit" means that no MIME-encoding is applied and its
syntax is in 7-bit form. MIME defines that "7bit" MIME-encoding is
assumed if content transfer encoding is not present. So, this field
can be omitted.
In certain situation, Base64 or Quoted-Printable MAY be used.
Base64 is RECOMMENDED to MIME-encode ISO-2022-JP text rather than
Quoted-Printable. This is because Base64 encoding is considered
more suitable for ISO-2022-JP text than Quoted-Printable encoding
for the following reasons: (1) Quoted-Printable encoding is designed
for text which is mostly ASCII but ISO-2022-JP text is NOT. (2)
Quoted-Printable encoding produces various results while Base64
encoding creates a unique output if folding is ignored. This means
that it is easier to implement interoperability for Base64 encoding
than for Quoted-Printable encoding.
IMPLEMENTATION NOTE: The key word "=" of Quoted-Printable encoding
is used in JIS X 0208 (see 'double-byte-char-e'). If only the ASCII
portion of ISO-2022-JP text is MIME-encoded with Quoted-Printable,
it is likely that such text cannot be MIME-decoded to the original
since the "=" characters in JIS X 0208 portion are considered to be
the key word of Quoted-Printable. If ISO-2022-JP text needs to be
MIME-encoded with Quoted-Printable, the entire text must be
MIME-encoded. Some old implementations made this mistake.
MIME viewers MUST be able to handle ISO-2022-JP text even if it is
encoded with Base64 or Quoted-Printable. Also, MIME viewers SHOULD
be able to handle ISO-2022-JP text even if its MIME-encoding is
Yamamoto [Page 9]
Internet-Draft ISO-2022-JP CHARSET January 1999
specified as "8bit".
5.3 Header Extensions
In a header or in MIME content headers where `encoded-word's are
allowed, Japanese string (i.e. excluding CRLF) which consists of the
four graphic character sets MUST be in the form defined in RFC
2047[MIME]. For "charset", "ISO-2022-JP" (case-insensitive) SHOULD
be specified. ISO-2022-JP string MUST end with ASCII (see 'line-e').
Message composers SHOULD use "B" encoding for ISO-2022-JP string
while message viewers MUST be able to handle both "B" and "Q"
encoding.
IMPLEMENTATION NOTE: It should be noted that these requirements
above apply to any fields in MIME content headers. For example, RFC
2047 mechanism MUST be used to represent ISO-2022-JP string when
embedded in Content-Description: content header in a part of a
multipart.
ISO-2022-JP string MUST NOT be embedded directly in a header NOR in
MIME content headers. EUC-JP string and Shift_JIS string SHOULD NOT
be inserted even if they are in the form defined in RFC 2047.
The reason why ISO-2022-JP string and "B" encoding is recommended to
embed Japanese string in header fields is that RFC 1468 defines so.
For historical reasons, not many message viewers support EUC-JP,
Shift_JIS, and "Q" encoding,
The "language" extension defined in RFC 2231[PARAMETER] SHOULD NOT
be used without any special purposes.
5.4 MIME Parameter Extensions
To embed Japanese string (i.e. excluding CRLF) to a MIME parameter
value, the string MUST be in the form defined in RFC 2231. For
"charset", "ISO-2022-JP" (case-insensitive) SHOULD be specified.
"language" SHOULD be omitted. The entire ISO-2022-JP string MUST
end with ASCII.
Both "extended-value" and "extended-other-values" need not to be
self-composite since they are to be concatenated first when
received. That is, they can contain any portion of ISO-2022-JP
string which may or may not end with ASCII.
IMPLEMENTATION NOTE: The MIME encoding defined in RFC 2231 is a
variant of "Q" encoding. Since many key words of header (including
"%" and "'") may appear in ISO-2022-JP string, they MUST be encoded
with the variant. It is a good idea to encode all characters other
than [0-9A-Za-z] (%d48-57 / %d65-90 / %d97-122).
6. Requirements for Message Transfer Agents (MTA)
Several MTAs automatically convert any message to a local charset.
Yamamoto [Page 10]
Internet-Draft ISO-2022-JP CHARSET January 1999
This means that it is very likely that a charset other than
ISO-2022-JP is converted to the local charset by a routine which
assumes that incoming data is ISO-2022-JP text. MTAs SHOULD NOT
convert charsets in general. If MTAs need to convert charsets, they
MUST do so according to the charset parameter of MIME messages.
Actually, some MTAs convert "ESC ( B" into "ESC ( J" and "ESC $ B"
into "ESC $ @" interchangeably. Gateways SHOULD NOT convert in this
way. Such MTAs are the primary cause of verification failure in
digital signature.
7. Historical Note
The JUNET code was originally described in the JUNET User's Guide
[JUNET].
JIS X 0208, which was originally called JIS C 6226, was published in
1978. It was revised in 1983, 1990 and 1997.
The revision in 1983 was made because the Japanese Government
published the new Kanji graphic character set called "Joyo Kanji Hyo
(Kanji List for Daily Use)" for use in daily life, in the public
documents, in newspapers in 1981. Some of the character shapes were
modified or simplified. To harmonize JIS X 0208 1978 with Joyo
Kanji, pairs of characters were swapped in the code positions and
several characters had their shapes amended. Moreover, since new
characters were added to the Kanji set called "Jinmei You Kanji
Beppyo (Additional Kanji List for Use with People's Name)", four
characters were added to and a lot of special characters were
included in JIS X 0208 1983. This graphic character set was treated
as a new one and a new escape sequence "ESC $ B" was assigned.
Since the Kanji set for People's names was extended, two more Kanji
characters were added at the end of JIS X 0208 in 1990 and the
result is called JIS X 0208 1990. The revised graphic character set
is indeed backward compatible with the previous set. Leaving the
designation sequence unchanged, only the revision identification
sequence "ESC & @" was assigned.
To be strict, the designation sequence of JIS X 0208 1990 is "ESC &
@ ESC $ B". However, ISO-2022-JP CES uses "ESC $ B" for practical
reasons even though the two additional Kanji characters are used.
The purpose of revision in 1997 is to clarify references of each
character. No new characters were introduced. As a graphic
character set, JIS X 0208 1997 is equivalent to JIS X 0208 1990.
The designation sequence is the same.
Therefore, among versions of JIS X 0208, the version of 1978 is of a
different graphic character set while the versions of 1983, 1990 and
1997 are regarded as a family.
For more information on the differences, please refer to Appendix J,
[GLOBEFISH]. It describes what characters have been added, swapped,
Yamamoto [Page 11]
Internet-Draft ISO-2022-JP CHARSET January 1999
simplified, etc, in the revisions.
JIS X 0201, originally called JIS C 6220, was published in 1976. It
was revised in 1997 but no change was introduced. They are thus the
same graphic character set.
8. Security Considerations
ISO-2022-JP is NOT believed to introduce any new security holes to
the Internet.
References
[ABNF] D. Crocker and Paul Overell, "Augmented BNF for Syntax
Specifications: ABNF", RFC2234, November 1997.
[ASCII] American National Standards Institute, "Coded character set
-- 7-bit American national standard code for information
interchange", ANSI X3.4-1986.
[CHARSET] N. Freed and J. Postel, "IANA Charset Registration
Procedures", RFC2278, January 1998.
[GLOBEFISH] K. Lunde, "Understanding Japanese Information
Processing", O'Reilly & Associates, Inc., 1993.
[GUIDE] G. Scott, "Guide for Internet Standards Writers", RFC2360,
June 1998.
[ISO-2022-JP-2] M. Ohta and K. Handa, "ISO-2022-JP-2: Multilingual
Extension of ISO-2022-JP", RFC 1554, December 1993.
[ISO2022] International Organization for Standardization (ISO),
"Information technology -- Character code structure and
extension techniques", International Standard, Ref. No. ISO
2022-1991(E).
[ISO646] International Organization for Standardization (ISO),
"Information technology -- ISO 7-bit coded character set for
information interchange", International Standard,
Ref. No. ISO/IEC 646:1991(E).
[ISOREG] International Organization for Standardization (ISO),
"International Register of Coded Character Sets To Be Used With
Escape Sequences".
[JISX0201-76] Japanese Industrial Standards Committee, "Code for
information interchange", JIS C 6220 1976(aka JIS X 0201 1976).
[JISX0201-97] Japanese Industrial Standards Committee, "7-bit and
8-bit coded character sets for information interchange", JIS X
0201 1997.
[JISX0208-78] Japanese Industrial Standards Committee, "Code of the
Yamamoto [Page 12]
Internet-Draft ISO-2022-JP CHARSET January 1999
Japanese graphic character set for information interchange", JIS
C 6226 1978(aka JIS X 0208 1978).
[JISX0208-83] Japanese Industrial Standards Committee, "Code of the
Japanese graphic character set for information interchange", JIS
C 6226 1983(aka JIS X 0208 1983).
[JISX0208-90] Japanese Industrial Standards Committee, "Code of the
Japanese graphic character set for information interchange", JIS
X 0208 1990.
[JISX0208-97] Japanese Industrial Standards Committee, "7-bit and
8-bit double byte coded KANJI sets for information interchange",
JIS X 0208 1997
[JISX0212-90] Japanese Industrial Standards Committee, "Code of the
supplementary Japanese graphic character set for information
interchange", JIS X0212 1990.
[JUNET] JUNET Riyou No Tebiki Sakusei Iin Kai (JUNET User's Guide
Drafting Committee), "JUNET Riyou No Tebiki (Dai Ippan)" ("JUNET
User's Guide (First Edition)"), February 1988.
[KEYWORDS] S. Bradner, "Key words for use in RFCs to Indicate
Requirement Levels", RFC 2119, March 1997.
[MAIL] D. Crocker, "Standard for the Format of ARPA Internet Text
Messages", STD 11, RFC 822, August 1982.
[MAIL2] P. Resnick, "Internet Message Format Standard",
draft-ietf-drums-msg-fmt-07.txt, January 1999.
[MIME] The primary definition of MIME. "MIME Part 1: Format of
Internet Message Bodies", RFC 2045; "MIME Part 2: Media Types",
RFC 2046; "MIME Part 3: Message Header Extensions for Non-ASCII
Text", RFC 2047; "MIME Part 4: Registration Procedures", RFC
2048; "MIME Part 5: Conformance Criteria and Examples", RFC 2049;
November 1996.
[NETNEWS] M. Horton and R. Adams, "Standard for Interchange of
USENET Messages", RFC 1036, December 1987.
[NNTP] B. Kantor and P. Lapsley, "Network News Transfer Protocol",
RFC 977, February 1986.
[OLDSPEC] J. Murai, M. Crispin, and E. van der Poel, "Japanese
Character Encoding for Internet Messages", RFC 1468, June 1993.
[PARAMETER] N. Freed and K. Moore, "MIME Parameter Value and Encoded
Word Extensions: Character Sets, Languages, and Continuations",
RFC 2231, November 1997.
[SMTP] J. Postel, "Simple Mail Transfer Protocol", RFC 821, August
1982.
Yamamoto [Page 13]
Internet-Draft ISO-2022-JP CHARSET January 1999
Acknowledgements
Many people contributed to RFC 1468. The original authors of RFC
1468 wished to thank in particular Akira Kato, Masahiro Sekiguchi
and Ken'ichi Handa.
The authors of this memo would acknowledge the original work of Erik
M. van der Poel and Mark Crispin. Our deep gratitude goes to
Noritoshi Demizu, Kenichi Handa, Jun'ichiro Ito, Shuhei Kobayashi,
Tomohiko Morioka, Makoto Murata, Erik M. van der Poel, Shigeya
Suzuki, and Akira Tanaka (in alphabetical order) for their
contribution to this memo.
Authors' Addresses
Eiiti WADA
Fujitsu Laboratories, Ltd
4-1-1 Kamikodanaka, Nakahara-ku, Kawasaki 211-8588 JAPAN
Phone: +81-44-754-2608
FAX: +81-44-754-2580
EMail: wada@u-tokyo.ac.jp
Jun MURAI
Keio University
5322 Endo, Fujisawa 252-0816 JAPAN
Phone: +81-466-47-5111
FAX: +81-466-49-1101
EMail: jun@wide.ad.jp
Kazuhiko YAMAMOTO
Research Laboratory, Internet Initiative Japan Inc.
Takebashi Yasuda Bldg., 3-13 Kanda Nishiki-cho Chiyoda-ku, Tokyo
101-0054 JAPAN
Phone: +81-3-5259-6350
FAX: +81-3-5259-6351
EMail: kazu@iijlab.net
Appendix
An example of a MIME text object is as follows:
Content-Type: text/plain; charset=iso-2022-jp
Content-Transfer-Encoding: 7bit
ISO-2022-JP text comes here.
Yamamoto [Page 14]
Internet-Draft ISO-2022-JP CHARSET January 1999
Another form of the example above is as follows:
Content-Type: Text/Plain; Charset=ISO-2022-JP
ISO-2022-JP text comes here.
The following is an example of RFC 2047 header encoding:
To: jun@wide.ad.jp
Subject: =?iso-2022-jp?B?GyRCJDMkTkZ8S1w4bCQsRkkkYSRsJFAbKEI=?=
OK
=?iso-2022-jp?B?GyRCJEckOSEjGyhC?=
From: Kazu Yamamoto (=?iso-2022-jp?B?GyRCOzNLXE9CSScbKEI=?=)
The following is an example of RFC 2231 parameter encoding:
Content-Disposition: attachment;
filename*=iso-2022-jp''%1B%24B%25U%25%21%25%24%25k%1B%28B
Change Log
Differences between 01 and 02
- Clarifies that %d0-31 is always the character set of ASCII.
- ISO-2022-JP is RECOMMENDED for MIME parameter extensions
according to suggestions from some vendors.
- The word "Character Encoding Scheme(CES)" is adopted to
align to RFC2278.
- Several typos are fixed.
- Message viewers MUST handle any combinations of %d0-255,
which was described as %d0-127.
- single-byte-char-e conforms [MAIL2].
- Refers to RFC 2046 to explain one reason to discourage use of
JIS X 0201.
Differences between 00 and 01
- Author information was updated.
- References were updated.
- Refers to the globefish book to describes the differences
between each revision.
- Explicitly says that ISO-2022-JP is syntactically and
semantically larger than US-ASCII.
- line-d and line-d are limited to 998.
- Clarifies that message viewers MUST handle any byte combinations.
- Includes the consideration for bare CR and bare LF.
Yamamoto [Page 15]