Chapter 07 of 37

Tokens

On this page

When we write a C program, the code is made up of many small individual elements.

For example:

int age = 20;

Here, int, age, =, and 20 are different elements of the program.

These individual elements are called tokens.

A token is the smallest meaningful unit of a C program that the compiler can recognize.

In simple words, you can think of tokens as the building blocks of a C program.

C tokens are mainly divided into six categories:

Let's understand them briefly.

Keywords are reserved words with a predefined meaning in C.

int age;
return 0;

Here, int and return are keywords.

Identifiers are names given to variables, functions, arrays, and other program elements.

int age = 20;

Here, age is an identifier.

Constants are fixed values that don't change during program execution.

int age = 20;

Here, 20 is a numeric constant.

Other examples include:

100
3.14
'A'

A sequence of characters written inside double quotes is called a string literal.

printf("Hello");

Here:

"Hello"

is a string literal.

Operators are symbols used to perform operations.

For example:

int sum = a + b;

Here, = and + are operators.

Some common operators are:

+   -   *   /
=   ==  >   <

C also uses various symbols to structure the program.

For example:

int main()
{
    printf("Hello");
}

Symbols such as:

( )   { }   ;   [ ]

are important parts of the C language syntax.

Example of Tokens

Consider this statement:

int age = 20;

We can break it into tokens:

int    → Keyword
age    → Identifier
=      → Operator
20     → Constant
;      → Special Symbol

So, a C program is basically built by combining these different types of tokens according to the rules of the language.

Tokens are the smallest meaningful building blocks of a C program.

The main categories you should remember are:

Keywords + Identifiers + Constants + String Literals + Operators + Special Symbols

Once you understand tokens, you'll have a much clearer idea of how the compiler looks at your C code.