In this article, we’ll take a look at using the strtok() and strtok_r() functions in C.

These functions are very useful, if you want to tokenize a string. C provides these handy utility functions to split our input string into tokens.

Let’s take a look at using these functions, using suitable examples.

Using the strtok() function

First, let’s look at the strtok() function.

This function is a part of the <string.h> header file, so you must include it in your program.

#include <string.h>

char* strtok(char* str, const char* delim);

This takes in an input string str and a delimiter character delim.

strtok() will split the string into tokens based on the delimited character.

We expect a list of strings from strtok(). But the function returns us a single string! Why is this?

The reason is how the function handles the tokenization. After calling strtok(input, delim), it returns the first token.

But we must keep calling the function again and again on a NULL input string, until we get NULL!

Basically, we need to keep calling strtok(NULL, delim) until it returns NULL.

Seems confusing? Let’s look at an example to clear it out!

#include <stdio.h>
#include <string.h>

int main() {
    // Our input string
    char input_string[] = "Hello from JournalDev!";

    // Our output token list
    char token_list[20][20]; 

    // We call strtok(input, delim) to get our first token
    // Notice the double quotes on delim! It is still a char* single character string!
    char* token = strtok(input_string, " ");

    int num_tokens = 0; // Index to token list. We will append to the list

    while (token != NULL) {
        // Keep getting tokens until we receive NULL from strtok()
        strcpy(token_list[num_tokens], token); // Copy to token list
        num_tokens++;
        token = strtok(NULL, " "); // Get the next token. Notice that input=NULL now!
    }

    // Print the list of tokens
    printf("Token List:\n");
    for (int i=0; i < num_tokens; i++) {
        printf("%s\n", token_list[i]);
    }

    return 0;
}

#c programming #c #strtok_r() #strtok()