Chapter 07 of 39

BCD, Gray Code, ASCII & Unicode

So far we have stored numbers in the most compact way possible: pure binary, 2's complement and IEEE 754 floating-point. These are perfect for arithmetic. But a computer does many things besides arithmetic. It shows numbers on a digital clock, reads the position of a rotating shaft, sends letters over a network and displays text in Hindi, Tamil and English on the same screen.

For each of these jobs a different code works better than plain binary:

  • BCD stores each decimal digit separately, so converting to and from human-readable decimal is trivial and decimal fractions like 0.1 are exact.

  • Gray code changes only one bit between consecutive values, so a sensor never reports a wildly wrong value while bits are changing.

  • ASCII and Unicode assign numbers to characters so that text can be stored and exchanged.

A binary code is a rule that maps each item of a set (digits, characters, positions) to a unique pattern of bits. The same bit pattern can mean different things under different codes, so the code must be agreed in advance.

For example, the 8 bits 0100 0001 are 65 in unsigned binary, 41 in BCD and the letter 'A' in ASCII. In this chapter we will learn each of these codes, do the conversions and BCD arithmetic step by step, and see how UTF-8 encodes characters such as ₹ and अ. All conversions were checked with Python and a C program.


Binary Coded Decimal (BCD)

In BCD (also called 8421 code), each decimal digit is encoded separately in 4 bits using its normal binary value. The bit weights are 8, 4, 2, 1.

Decimal digit

BCD (8421)

Decimal digit

BCD (8421)

0

0000

5

0101

1

0001

6

0110

2

0010

7

0111

3

0011

8

1000

4

0100

9

1001

The six patterns 1010 to 1111 are invalid in BCD; they never appear in a correct BCD number.

BCD vs Binary

Decimal 2026

BCD:     0010  0000  0010  0110        (16 bits, one group per digit)
           2     0     2     6

Binary:  111 1110 1010                 (11 bits)

Notice the difference: BCD converts digit by digit, binary converts the whole number. Students often mix them up. 15 in BCD is 0001 0101, but 15 in binary is 1111.

Point

Pure binary

BCD

Storage

Compact (n bits give 2n values)

Wasteful (4 bits give only 10 values; 16 bits store 0 to 9999 instead of 0 to 65535)

Arithmetic

Simple and fast

Needs a correction step

Conversion to decimal for display

Needs repeated division by 10

Trivial: each nibble is one digit

Decimal fractions like 0.10

Not exact

Exact

Typical uses

General computing

Digital clocks, calculators, 7-segment displays, real-time clock chips, financial/decimal arithmetic

Banking software cannot accept that ₹0.10 is stored as 0.1000000000000000055 (as we saw in Floating-Point Representation (IEEE 754)). That is why decimal formats survive: IBM mainframes have hardware decimal arithmetic, and the 2008 revision of IEEE 754 added decimal floating-point formats.

Packed and Unpacked BCD

  • Unpacked BCD: one digit per byte (upper nibble 0000). 47 = 0000 0100 0000 0111.

  • Packed BCD: two digits per byte. 47 = 0100 0111 (0x47).

The 8085 and x86 processors have instructions to adjust results after binary addition of packed BCD (DAA, "decimal adjust after addition"). On x86 these instructions are not available in 64-bit mode, a sign of how rarely they are used today.

BCD Addition

If we add two BCD digits with an ordinary binary adder, the result is correct as long as it is 9 or less. But if the sum is 10 to 19, the 4-bit result is either an invalid pattern (1010 to 1111) or produces a carry that skipped six patterns. The fix:

BCD addition rule: add the digits in binary. If the 4-bit sum is greater than 9 (1001) or a carry came out of the group, add 6 (0110) to that group. The carry from the correction goes to the next digit.

Why 6? A 4-bit group counts up to 16 before carrying, but a decimal digit should carry at 10. The six unused patterns (1010 to 1111) are the gap. Adding 6 skips over them, forcing the carry to happen at the right point.

Example 1: 58 + 67 in BCD

         tens   units
  58  =  0101   1000
  67  =  0110   0111

Units:  1000 + 0111 = 1111  (15 > 9, invalid)
        1111 + 0110 = 1 0101    → digit 5, carry 1

Tens:   0101 + 0110 + 1 (carry) = 1100  (12 > 9, invalid)
        1100 + 0110 = 1 0010    → digit 2, carry 1

Result: 0001 0010 0101 = 125  ✓

Example 2: 9 + 9 (the Carry Case)

1001 + 1001 = 1 0010     ← looks like a valid digit 2 with a carry,
                           but the answer should be 18, not 12

The carry came out of the group, so correct it:
0010 + 0110 = 1000, and keep the carry 1

Result: 0001 1000 = 18  ✓

This is why the rule checks both conditions: sum > 9, or a carry out. A circuit for one BCD digit (a "BCD adder") is a 4-bit binary adder, a detector Cout = K + Z8Z4 + Z8Z2 (K is the binary carry, Z the binary sum bits), and a second adder that adds 0110 when Cout = 1. (Adders themselves are built in Combinational Circuits: Adders, Multiplexers & Decoders.)


Other Decimal Codes: Excess-3 and 2421

Excess-3 (XS-3)

Excess-3 code represents each decimal digit d by the 4-bit binary value of d + 3.

It is a non-weighted code (bit positions have no fixed weights). Its useful property is that it is self-complementing: the 9's complement of a digit is obtained by simply inverting all four bits. For example, 2 is 0101 in XS-3; invert it to get 1010, which is 7 in XS-3, and 9 − 2 = 7. This made subtraction (via 9's complement) easy in early decimal machines.

2421 (Aiken) Code

A weighted code with weights 2, 4, 2, 1. Some digits have two possible patterns; the standard choice (0 to 4 start with 0, 5 to 9 start with 1) is also self-complementing.

Digit

BCD 8421

Excess-3

2421

0

0000

0011

0000

1

0001

0100

0001

2

0010

0101

0010

3

0011

0110

0011

4

0100

0111

0100

5

0101

1000

1011

6

0110

1001

1100

7

0111

1010

1101

8

1000

1011

1110

9

1001

1100

1111

Check the self-complementing property in the table: in both the excess-3 and 2421 columns, the rows for d and 9 − d are bitwise complements (e.g. 2421: 3 = 0011 and 6 = 1100). BCD 8421 is not self-complementing (3 = 0011 but 6 = 0110).


Gray Code

Imagine a sensor that reads the angle of a rotating disc as a 3-bit binary number. Going from 3 (011) to 4 (100), all three bits change. Physical contacts never switch at exactly the same instant, so for a moment the sensor might read 111 (7) or 000 (0), a completely wrong position.

Gray code (reflected binary code) is an unweighted code in which successive values differ in exactly one bit. It is also cyclic: the last value differs from the first in one bit.

With Gray code, a reading taken mid-change is always either the old value or the new value, never a third, unrelated one.

Building Gray Code by Reflection

1-bit:  0, 1

2-bit:  write the 1-bit list, then its mirror image;
        prefix the first half with 0 and the second half with 1
        00, 01 | 11, 10

3-bit:  000, 001, 011, 010 | 110, 111, 101, 100

4-bit Binary and Gray Code Table

Decimal

Binary

Gray

Decimal

Binary

Gray

0

0000

0000

8

1000

1100

1

0001

0001

9

1001

1101

2

0010

0011

10

1010

1111

3

0011

0010

11

1011

1110

4

0100

0110

12

1100

1010

5

0101

0111

13

1101

1011

6

0110

0101

14

1110

1001

7

0111

0100

15

1111

1000

Binary to Gray Conversion

The most significant Gray bit equals the most significant binary bit. Every other Gray bit is the XOR of the corresponding binary bit and the binary bit to its left: gi = bi+1 ⊕ bi. In code: gray = b ^ (b >> 1).

Binary 1011 → Gray

b:   1    0    1    1
     |  \ |  \ |  \ |
g:   1   1⊕0  0⊕1  1⊕1
     1    1    1    0         Gray = 1110

Gray to Binary Conversion

The most significant binary bit equals the most significant Gray bit. Every other binary bit is the XOR of the corresponding Gray bit and the binary bit just computed to its left: bi = bi+1 ⊕ gi.

Gray 1101 → Binary

g:   1     1     0     1
b:   1   1⊕1=0  0⊕0=0  0⊕1=1      Binary = 1001  (decimal 9)

Check in the table: 9 → Gray 1101  ✓

The important difference: binary→Gray XORs neighbouring input bits; Gray→binary XORs each Gray bit with the previous output bit, so it is a running XOR from the left.

Where Gray Code Is Used

  • Rotary and linear position encoders: the original motivation.

  • Karnaugh maps: the rows and columns are labelled in Gray order (00, 01, 11, 10) so that physically adjacent cells differ in one variable. You will use this in the next chapter, Logic Gates & Boolean Algebra.

  • Crossing clock domains: asynchronous FIFOs pass read/write pointers between clock domains in Gray code, so a pointer sampled mid-change is off by at most one.

  • Low-power counters: only one bit toggles per count, reducing switching.

Gray code is not suitable for arithmetic: there are no simple positional weights.


Error Detection with a Parity Bit

When bits travel over a wire or sit in memory, one might flip. The simplest protection is a parity bit added to each code word:

  • Even parity: the parity bit makes the total number of 1s even.

  • Odd parity: the parity bit makes the total number of 1s odd.

Data (7 bits)   Number of 1s   Even-parity bit   Odd-parity bit
1011001             4                0                 1
1110000             3                1                 0

The receiver counts the 1s again. If the parity is wrong, an error happened. The even-parity bit is just the XOR of all data bits, so a parity generator or checker is a tree of XOR gates. Parity detects any odd number of flipped bits but misses any even number, and it cannot tell which bit is wrong. Codes that can correct errors (such as Hamming codes and the ECC used in server memory) add several check bits.


ASCII

ASCII (American Standard Code for Information Interchange) is a 7-bit character code with 128 code points (0 to 127): 33 control characters (0 to 31 and 127) and 95 printable characters (32 to 126).

Characters

Decimal

Hex

Binary

NUL (null, ends C strings)

0

00

000 0000

TAB, LF (newline), CR

9, 10, 13

09, 0A, 0D

-

Space

32

20

010 0000

'0' to '9'

48 to 57

30 to 39

011 0000 to 011 1001

'A' to 'Z'

65 to 90

41 to 5A

100 0001 to 101 1010

'a' to 'z'

97 to 122

61 to 7A

110 0001 to 111 1010

DEL

127

7F

111 1111

The layout was designed carefully, and it pays to know why:

  • Digits: '0' is 0x30, so the low nibble of a digit character is its value. c - '0' converts a digit character to its number. (This is also why ASCII digits are "almost BCD".)

  • Letters: upper and lower case differ by exactly 32, which is bit 5 (0x20). 'A' = 100 0001, 'a' = 110 0001. Setting bit 5 converts to lower case, clearing it converts to upper case.

  • Order: letters are in alphabetical order and digits in numeric order, so string comparison by codes gives a sensible sort (with all capitals before all lower-case letters).

Since bytes are 8 bits, the spare eighth bit was once used for parity, and later for extended ASCII code pages (such as ISO 8859-1 for Western European languages) that defined characters 128 to 255 differently in each region. A file written in one code page looked like garbage in another. IBM mainframes used an unrelated 8-bit code, EBCDIC. India had its own standard, ISCII, for Indian scripts.


Unicode

256 characters are not enough for even one Indian state's languages, let alone Chinese, Japanese, Arabic and emoji. Unicode solves this with one universal character set.

Unicode assigns every character a unique number called a code point, written U+ followed by at least four hex digits. Code points range from U+0000 to U+10FFFF, a space of 1,114,112 code points organized as 17 planes of 65,536 each.

  • Plane 0, U+0000 to U+FFFF, is the Basic Multilingual Plane (BMP). It contains almost all characters in everyday use, including Devanagari (U+0900 to U+097F), Tamil, Bengali and other Indian scripts.

  • The first 128 code points are identical to ASCII, so 'A' is U+0041.

  • Emoji and many historic scripts are in the supplementary planes, e.g. 😀 is U+1F600.

A code point is just a number. To store it in bytes we need an encoding. Unicode defines three encoding forms: UTF-8, UTF-16 and UTF-32.

UTF-8

UTF-8 uses 1 to 4 bytes per code point:

Code point range

Bytes

Byte pattern (x = code point bits)

Payload bits

U+0000 to U+007F

1

0xxxxxxx

7

U+0080 to U+07FF

2

110xxxxx 10xxxxxx

11

U+0800 to U+FFFF

3

1110xxxx 10xxxxxx 10xxxxxx

16

U+10000 to U+10FFFF

4

11110xxx 10xxxxxx 10xxxxxx 10xxxxxx

21

The design is clever:

  • ASCII compatible: every ASCII file is already valid UTF-8, byte for byte.

  • Self-synchronizing: continuation bytes always start with 10, and a leading byte never does. If you jump into the middle of a string, you can find the start of the next character.

  • No zero bytes except for U+0000 itself, so C string functions keep working.

  • No byte-order problem: it is a sequence of bytes, so endianness does not matter.

UTF-8 is used by the overwhelming majority of web pages and is the default on Linux.

Example 3: Encode ₹ (U+20B9) in UTF-8

U+20B9 lies in U+0800..U+FFFF  → 3 bytes, 16 payload bits

0x20B9 = 0010 0000 1011 1001

Split into 4 + 6 + 6 bits:   0010 | 000010 | 111001

Fill the template:
   1110 xxxx   10 xxxxxx   10 xxxxxx
   1110 0010   10 000010   10 111001
   = E2         82          B9

So ₹ is stored as the three bytes E2 82 B9. In the same way, Devanagari अ (U+0905) becomes E0 A4 85, and 😀 (U+1F600) becomes the four bytes F0 9F 98 80.

UTF-16 and UTF-32

  • UTF-32 stores every code point in 4 bytes. Simple (the n-th character is at byte 4n), but wasteful.

  • UTF-16 stores BMP characters in 2 bytes and supplementary characters in 4 bytes as a surrogate pair. The range U+D800 to U+DFFF (2048 code points) is reserved for surrogates and never assigned to characters. Java, JavaScript and Windows use UTF-16 internally.

Surrogate pair for U+1F600: subtract 0x10000 to get 0xF600, a 20-bit value 0000111101 1000000000. The high 10 bits (0x03D) are added to 0xD800 and the low 10 bits (0x200) to 0xDC00, giving D83D DE00.

Because UTF-16 and UTF-32 use multi-byte units, the byte order matters (little-endian or big-endian). A file may begin with a byte order mark (U+FEFF) to tell the reader which order was used.

Character

Code point

UTF-8

UTF-16 (BE)

UTF-32 (BE)

A

U+0041

41

00 41

00 00 00 41

é

U+00E9

C3 A9

00 E9

00 00 00 E9

अ

U+0905

E0 A4 85

09 05

00 00 09 05

₹

U+20B9

E2 82 B9

20 B9

00 00 20 B9

😀

U+1F600

F0 9F 98 80

D8 3D DE 00

00 01 F6 00

Characters, Code Points and Bytes Are Different Things

The word नमस्ते looks like 3 or 4 letters to a reader, but it is 6 code points (न म स ् त े, where ् and े are combining signs), 18 bytes in UTF-8 and 12 bytes in UTF-16. "Length" depends on what you count. A C program shows the byte view clearly (compiled with GCC, source saved as UTF-8):

#include <stdio.h>
#include <string.h>

int main(void) {
    char c = '7';
    printf("'7' has code %d, numeric value %d\n", c, c - '0');
    printf("'A' | 0x20 = '%c', 'g' & ~0x20 = '%c'\n", 'A' | 0x20, 'g' & ~0x20);

    const char *s = "₹100";              /* source file saved as UTF-8 */
    printf("strlen(\"₹100\") = %zu bytes:", strlen(s));
    for (size_t i = 0; i < strlen(s); i++)
        printf(" %02X", (unsigned char)s[i]);
    printf("\n");
    return 0;
}
'7' has code 55, numeric value 7
'A' | 0x20 = 'a', 'g' & ~0x20 = 'G'
strlen("₹100") = 6 bytes: E2 82 B9 31 30 30

strlen counts bytes, not characters: the single symbol ₹ takes 3 bytes.


All the Codes at a Glance

Code

Weighted?

Key property

Main use

BCD (8421)

Yes (8, 4, 2, 1)

One nibble per decimal digit

Displays, clocks, decimal arithmetic

Excess-3

No

Self-complementing

Early decimal arithmetic units

2421

Yes (2, 4, 2, 1)

Self-complementing

Decimal arithmetic

Gray

No

Neighbours differ in one bit; cyclic

Encoders, K-maps, clock-domain crossing

ASCII

-

7 bits, 128 characters

English text, protocols

UTF-8

-

1 to 4 bytes, ASCII compatible

Web, files, Linux

UTF-16

-

2 or 4 bytes (surrogates)

Java, JavaScript, Windows APIs


Common Mistakes

  • Converting the whole number to binary when BCD is asked. BCD is digit by digit: 93 = 1001 0011, not 101 1101.

  • Forgetting the carry case in BCD addition. Add 6 when the group sum is above 9 or produced a carry (9 + 9 = 1 0010 needs correction).

  • Doing Gray→binary like binary→Gray. Gray→binary XORs with the previously computed binary bit, not with the neighbouring Gray bit.

  • Calling Gray code weighted. It is unweighted and not used for arithmetic.

  • Saying ASCII is 8 bits. Standard ASCII is 7 bits (128 characters); 8-bit "extended ASCII" is a family of incompatible code pages.

  • Saying Unicode is a 16-bit code. Code points go up to U+10FFFF (21 bits). UTF-16 needs surrogate pairs beyond U+FFFF.

  • Confusing Unicode (the character set) with UTF-8 (one encoding of it).


GATE and Interview Questions

Q1. Represent 395 in BCD and in binary. How many bits does each need?

BCD: 0011 1001 0101 (12 bits). Binary: 395 = 256 + 128 + 8 + 2 + 1 = 1 1000 1011 (9 bits).

Q2. Add 386 and 475 in BCD.

Units: 0110 + 0101 = 1011 (> 9) + 0110 = 1 0001 → digit 1, carry 1. Tens: 1000 + 0111 + 1 = 1 0000 (carry out) + 0110 = 1 0110 → digit 6, carry 1. Hundreds: 0011 + 0100 + 1 = 1000 → digit 8, no correction. Result: 1000 0110 0001 = 861.

Q3. Convert binary 101101 to Gray code, and Gray code 111011 to binary.

Binary→Gray: 1, 1⊕0=1, 0⊕1=1, 1⊕1=0, 1⊕0=1, 0⊕1=1 → 111011. Gray→binary: 1, 1⊕1=0, 0⊕1=1, 1⊕0=1, 1⊕1=0, 0⊕1=1 → 101101. (They are inverses of each other, as expected.)

Q4. Why is Gray code used in K-maps and shaft encoders?

Because successive values differ in only one bit. In K-maps this makes adjacent cells differ in one variable, so groups can be simplified. In encoders it guarantees that a reading taken while bits are changing is off by at most one position.

Q5. Which of BCD, excess-3 and 2421 are self-complementing?

Excess-3 and 2421. In them the code of 9 − d is the bitwise complement of the code of d. BCD 8421 is not.

Q6. How do you convert a lowercase ASCII letter to uppercase using a bit operation?

Clear bit 5: c & ~0x20 (equivalently c & 0xDF). Upper and lower case letters differ only in that bit (difference 32).

Q7. How many bytes does UTF-8 use for U+0905 (Devanagari अ), and what are they?

U+0905 is in the range U+0800 to U+FFFF, so 3 bytes. 0x0905 = 0000 1001 0000 0101 → 0000 | 100100 | 000101 → 11100000 10100100 10000101 = E0 A4 85.

Q8. Why did UTF-8 become the dominant encoding?

It is backward compatible with ASCII, has no byte-order issues, is compact for English and markup, never produces stray zero bytes, and is self-synchronizing.


Summary

  • BCD stores each decimal digit in 4 bits (8421). It wastes 6 of 16 patterns but makes decimal display and exact decimal arithmetic easy. In BCD addition, add 6 to any digit group whose sum is above 9 or that produced a carry.

  • Excess-3 (d + 3) and 2421 are self-complementing decimal codes.

  • Gray code changes one bit between consecutive values. Binary→Gray: gi = bi+1 ⊕ bi. Gray→binary: bi = bi+1 ⊕ gi (running XOR).

  • A parity bit detects any odd number of bit errors.

  • ASCII is a 7-bit, 128-character code; digits start at 0x30, 'A' at 0x41, 'a' at 0x61, and case is bit 5.

  • Unicode gives every character a code point from U+0000 to U+10FFFF; UTF-8 (1 to 4 bytes), UTF-16 (2 or 4) and UTF-32 (4) store them as bytes.

  • Next, in Logic Gates & Boolean Algebra, we move from how bits are encoded to how hardware actually processes them.