GB 18030 - GB18030 As A Code Page

GB18030 As A Code Page

GB18030 is the registered Internet name for the official character set of the People's Republic of China (PRC) superseding GB2312. This character set is formally called "Chinese National Standard GB 18030-2005: Information technology — Chinese coded character set". GB abbreviates Guójiā Biāozhǔn (国家标准), which means national standard in Chinese. The standard was published by the China Standard Press, Beijing, November 8, 2005. Only a portion of the standard is mandatory. Since May 1, 2006, support for the mandatory subset is officially required for all software products sold in the PRC. Due to its Unicode equivalence, GB18030 supports both simplified and traditional Chinese characters.

An older version of the standard, known as "Chinese National Standard GB 18030-2000: Information Technology — Chinese ideograms coded character set for information interchange — Extension for the basic set", was published on March 17, 2000. The encoding scheme remains the same in the new version, except that code points for the characters ḿ and ɟ have been exchanged. More code points are now associated to characters due to update of Unicode, especially the appearance of CJK Unified Ideographs Extension B. Some characters used by ethnic minorities in China, such as Mongolian characters and Tibetan characters (GB 16959-1997 and GB/T 20542-2006), have been added as well, which accounts for the renaming of the standard.

GB18030 can be considered a Unicode Transformation Format (i.e. an encoding of all Unicode code points) that maintains compatibility with a legacy character set. Like UTF-8, GB18030 is a superset of ASCII and can represent the whole range of Unicode code points; in addition, it is also a superset of GB2312. GB18030 also maintains compatibility with Windows Codepage 936, sometimes known as GBK, which is Microsoft's extended version of GB2312, with the exception of the euro sign which is given a single byte code of 0x80 in Microsoft's later versions of GBK and a two byte code of A2 E3 in GB18030. GB 18030-2005 is also compatible with Chinese Internal Code Specification, Version 1.0, known as GBK 1.0, which is a slight extension of Windows Codepage 936 in 1995. Mapping to Unicode, however, has been modified for the 81 characters that were provisionally assigned a Unicode PUA code point in GBK 1.0 and that have later been encoded in Unicode. This is specified in Appendix E of GB 18030-2005. There are 14 characters in GB 18030-2005 that are still mapped to Unicode PUA.

Part of the mapping data is from a lookup table (similarly to GBK). The rest is calculated algorithmically. Unfortunately it also inherits the bad aspects of the legacy standards it's based on (most notably needing special code to safely find ASCII characters in a GB18030 sequence).

Most major computer companies had already standardised on some version of Unicode as the primary format for use in their binary formats and OS calls. However, they mostly had only supported code points in the BMP originally defined in Unicode 1.0, which supported only 65,536 codepoints and was often encoded in 16 bits as UCS-2.

The mandatory part of GB 18030-2005 consists of 1 byte and 2 byte encoding, together with 4 byte encoding for CJK Unified Ideographs Extension A. The corresponding Unicode code points of this subset lie entirely in the BMP.

In a move of historic significance for software supporting Unicode, the PRC decided to mandate support of certain code points outside the BMP. This means that software can no longer get away with treating characters as 16 bit fixed width entities (UCS-2). Therefore they must either process the data in a variable width format (such as UTF-8 or UTF-16), which are the most common choices, or move to a larger fixed width format (such as UCS-4 or UTF-32). Microsoft made the change from UCS-2 to UTF-16 with Windows 2000.