Concatenate number sequence with mathematical operator without executing the operator

I have to concatenate this DOI of a paper "10.1145/3175684.3175695" at the end of a URL&nbsp;<code><a href="" target="_blank"></a></code>. But when I try to do it, python concatenates the divided result. Is there a way to tell python to not treat the&nbsp;<code>/</code>&nbsp;symbol in the middle of&nbsp;<code>10.1145/3175684.3175695</code>&nbsp;as a division operator.

I have to concatenate this DOI of a paper "10.1145/3175684.3175695" at the end of a URL But when I try to do it, python concatenates the divided result. Is there a way to tell python to not treat the / symbol in the middle of 10.1145/3175684.3175695 as a division operator.

id = 10.1145/3175684.3175695
url = '{}{}'.format("",id)
# Make a get request with the parameters.
response = requests.get(url)

How to Write Python C Extension Modules using the Python API

How to Write Python C Extension Modules using the Python API

There are several ways in which you can extend the functionality of Python. One of these is to write your Python module in C or C++. In this tutorial, you’ll discover how to use the Python API to write Python C extension modules.

You’ll learn how to:

  • Invoke C functions from within Python
  • Pass arguments from Python to C and parse them accordingly
  • Raise exceptions from C code and create custom Python exceptions in C
  • Define global constants in C and make them accessible in Python
  • Test, package, and distribute your Python C extension module

Table of Contents

  • Extending Your Python Program
  • Writing a Python Interface in C
    • Understanding fputs()
    • Writing the C Function for fputs()
    • Wrapping fputs()
    • Writing the Init Function
    • Putting It All Together
  • Packaging Your Python C Extension Module
    • Building Your Module
    • Running Your Module
  • Raising Exceptions
    • Raising Exceptions From C Code
    • Raising Custom Exceptions
  • Defining Constants
  • Testing Your Module
  • Considering Alternatives
  • Conclusion
Extending Your Python Program

One of the lesser-known yet incredibly powerful features of Python is its ability to call functions and libraries defined in compiled languages such as C or C++. This allows you to extend the capabilities of your program beyond what Python’s built-in features have to offer.

There are many languages you could choose from to extend the functionality of Python. So, why should you use C? Here are a few reasons why you might decide to build a Python C extension module:

  1. To implement new built-in object types: It’s possible to write a Python class in C, and then instantiate and extend that class from Python itself. There can be many reasons for doing this, but more often than not, performance is primarily what drives developers to turn to C. Such a situation is rare, but it’s good to know the extent to which Python can be extended.

  2. To call C library functions and system calls: Many programming languages provide interfaces to the most commonly used system calls. Still, there may be other lesser-used system calls that are only accessible through C. The os module in Python is one example.

This is not an exhaustive list, but it gives you the gist of what can be done when extending Python using C or any other language.

To write Python modules in C, you’ll need to use the Python API, which defines the various functions, macros, and variables that allow the Python interpreter to call your C code. All of these tools and more are collectively bundled in the Python.h header file.

Writing a Python Interface in C

In this tutorial, you’ll write a small wrapper for a C library function, which you’ll then invoke from within Python. Implementing a wrapper yourself will give you a better idea about when and how to use C to extend your Python module.

Understanding fputs()

fputs() is the C library function that you’ll be wrapping:

int fputs(const char *, FILE *)

This function takes two arguments:

  1. const char * is an array of characters.
  2. FILE * is a file stream pointer.

fputs() writes the character array to the file specified by the file stream and returns a non-negative value. If the operation is successful, then this value will denote the number of bytes written to the file. If there’s an error, then it returns EOF. You can read more about this C library function and its other variants in the manual page entry.

Writing the C Function for fputs()

This is a basic C program that uses fputs() to write a string to a file stream:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

int main() {
    FILE *fp = fopen("write.txt", "w");
    fputs("Real Python!", fp);
    return 1;

This snippet of code can be summarized as follows:

  1. Open the file write.txt.
  2. Write the string "Real Python!" to the file.

Note: The C code in this article should build on most systems. It has been tested on GCC without using any special flags.

In the following section, you’ll write a wrapper for this C function.

Wrapping fputs()

It might seem a little weird to see the full code before an explanation of how it works. However, taking a moment to inspect the final product will supplement your understanding in the following sections. The code block below shows the final wrapped version of your C code:

static PyObject *method_fputs(PyObject *self, PyObject *args) {

    char *str, *filename = NULL;

    int bytes_copied = -1;

    /* Parse arguments */

    if(!PyArg_ParseTuple(args, "ss", &str, &filename)) {

        return NULL;


    FILE *fp = fopen(filename, "w");

    bytes_copied = fputs(str, fp);


    return PyLong_FromLong(bytes_copied);


This code snippet references three object structures:

  1. PyObject
  2. PyArg_ParseTuple()
  3. PyLong_FromLong()

These are used for data type definition for the Python language. You’ll go through each of them now.


PyObject is an object structure that you use to define object types for Python. All Python objects share a small number of fields that are defined using the PyObject structure. All other object types are extensions of this type.

PyObject tells the Python interpreter to treat a pointer to an object as an object. For instance, setting the return type of the above function as PyObject defines the common fields that are required by the Python interpreter in order to recognize this as a valid Python type.

Take another look at the first few lines of your C code:

static PyObject *method_fputs(PyObject *self, PyObject *args) {

    char *str, *filename = NULL;

    int bytes_copied = -1;

    /* Snip */

In line 2, you declare the argument types you wish to receive from your Python code:

  1. char *str is the string you want to write to the file stream.
  2. char *filename is the name of the file to write to.


PyArg_ParseTuple() parses the arguments you’ll receive from your Python program into local variables:

static PyObject *method_fputs(PyObject *self, PyObject *args) {

    char *str, *filename = NULL;

    int bytes_copied = -1;

    /* Parse arguments */

    if(!PyArg_ParseTuple(args, "ss", &str, &filename)) {

        return NULL;


    /* Snip */

If you look at line 6, then you’ll see that PyArg_ParseTuple() takes the following arguments:

  • args are of type PyObject.

  • "ss" is the format specifier that specifies the data type of the arguments to parse. (You can check out the official documentation for a complete reference.)

  • &str and &filename are pointers to local variables to which the parsed values will be assigned.

PyArg_ParseTuple() evaluates to false on failure. If it fails, then the function will return NULL and not proceed any further.


As you’ve seen before, fputs() takes two arguments, one of which is the FILE * object. Since you can’t parse a Python textIOwrapper object using the Python API in C, you’ll have to use a workaround:

static PyObject *method_fputs(PyObject *self, PyObject *args) {

    char *str, *filename = NULL;

    int bytes_copied = -1;

    /* Parse arguments */

    if(!PyArg_ParseTuple(args, "ss", &str, &filename)) {

        return NULL;


    FILE *fp = fopen(filename, "w");

    bytes_copied = fputs(str, fp);


    return PyLong_FromLong(bytes_copied);


Here’s a breakdown of what this code does:

  • In line 10, you’re passing the name of the file that you’ll use to create a FILE * object and pass it on to the function.
  • In line 11, you call fputs() with the following arguments:
    • str is the string you want to write to the file.
    • fp is the FILE * object you defined in line 10.

You then store the return value of fputs() in bytes_copied. This integer variable will be returned to the fputs() invocation within the Python interpreter.


PyLong_FromLong() returns a PyLongObject, which represents an integer object in Python. You can find it at the very end of your C code:

static PyObject *method_fputs(PyObject *self, PyObject *args) {

    char *str, *filename = NULL;

    int bytes_copied = -1;

    /* Parse arguments */

    if(!PyArg_ParseTuple(args, "ss", &str, &filename)) {

        return NULL;


    FILE *fp = fopen(filename, "w");

    bytes_copied = fputs(str, fp);


    return PyLong_FromLong(bytes_copied);


Line 14 generates a PyLongObject for bytes_copied, the variable to be returned when the function is invoked in Python. You must return a PyObject* from your Python C extension module back to the Python interpreter.

Writing the Init Function

You’ve written the code that makes up the core functionality of your Python C extension module. However, there are still a few extra functions that are necessary to get your module up and running. You’ll need to write definitions of your module and the methods it contains, like so:

static PyMethodDef FputsMethods[] = {
    {"fputs", method_fputs, METH_VARARGS, "Python interface for fputs C library function"},
    {NULL, NULL, 0, NULL}

static struct PyModuleDef fputsmodule = {
    "Python interface for the fputs C library function",

These functions include meta information about your module that will be used by the Python interpreter. Let’s go through each of the structs above to see how they work.


In order to call the methods defined in your module, you’ll need to tell the Python interpreter about them first. To do this, you can use PyMethodDef. This is a structure with 4 members representing a single method in your module.

Ideally, there will be more than one method in your Python C extension module that you want to be callable from the Python interpreter. This is why you need to define an array of PyMethodDef structs:

static PyMethodDef FputsMethods[] = {
    {"fputs", method_fputs, METH_VARARGS, "Python interface for fputs C library function"},
    {NULL, NULL, 0, NULL}

Each individual member of the struct holds the following info:

  • "fputs" is the name the user would write to invoke this particular function.

  • method_fputs is the name of the C function to invoke.

  • METH_VARARGS is a flag that tells the interpreter that the function will accept two arguments of type PyObject*:

    1. self is the module object.
    2. args is a tuple containing the actual arguments to your function. As explained previously, these arguments are unpacked using PyArg_ParseTuple().
  • The final string is a value to represent the method docstring.


Just as PyMethodDef holds information about the methods in your Python C extension module, the PyModuleDef struct holds information about your module itself. It is not an array of structures, but rather a single structure that’s used for module definition:

static struct PyModuleDef fputsmodule = {
    "Python interface for the fputs C library function",

There are a total of 9 members in this struct, but not all of them are required. In the code block above, you initialize the following five:

  1. PyModuleDef_HEAD_INIT is a member of type PyModuleDef_Base, which is advised to have just this one value.

  2. "fputs" is the name of your Python C extension module.

  3. The string is the value that represents your module docstring. You can use NULL to have no docstring, or you can specify a docstring by passing a const char * as shown in the snippet above. It is of type Py_ssize_t. You can also use PyDoc_STRVAR() to define a docstring for your module.

  4. -1 is the amount of memory needed to store your program state. It’s helpful when your module is used in multiple sub-interpreters, and it can have the following values:

    • A negative value indicates that this module doesn’t have support for sub-interpreters.
    • A non-negative value enables the re-initialization of your module. It also specifies the memory requirement of your module to be allocated on each sub-interpreter session.
  5. FputsMethods is the reference to your method table. This is the array of PyMethodDef structs you defined earlier.

For more information, check out the official Python documentation on PyModuleDef.


Now that you’ve defined your Python C extension module and method structures, it’s time to put them to use. When a Python program imports your module for the first time, it will call PyInit_fputs():

PyMODINIT_FUNC PyInit_fputs(void) {
    return PyModule_Create(&fputsmodule);

PyMODINIT_FUNC does 3 things implicitly when stated as the function return type:

  1. It implicitly sets the return type of the function as PyObject*.
  2. It declares any special linkages.
  3. It declares the function as extern “C.” In case you’re using C++, it tells the C++ compiler not to do name-mangling on the symbols.

PyModule_Create() will return a new module object of type PyObject *. For the argument, you’ll pass the address of the method structure that you’ve already defined previously, fputsmodule.

Note: In Python 3, your init function must return a PyObject * type. However, if you’re using Python 2, then PyMODINIT_FUNC declares the function return type as void.

Putting It All Together

Now that you’ve written the necessary parts of your Python C extension module, let’s take a step back to see how it all fits together. The following diagram shows the components of your module and how they interact with the Python interpreter:

When you import your Python C extension module, PyInit_fputs() is the first method to be invoked. However, before a reference is returned to the Python interpreter, the function makes a subsequent call to PyModule_Create(). This will initialize the structures PyModuleDef and PyMethodDef, which hold meta information about your module. It makes sense to have them ready since you’ll make use of them in your init function.

Once this is complete, a reference to the module object is finally returned to the Python interpreter. The following diagram shows the internal flow of your module:

The module object returned by PyModule_Create() has a reference to the module structure PyModuleDef, which in turn has a reference to the method table PyMethodDef. When you call a method defined in your Python C extension module, the Python interpreter uses the module object and all of the references it carries to execute the specific method. (While this isn’t exactly how the Python interpreter handles things under the hood, it’ll give you an idea of how it works.)

Similarly, you can access various other methods and properties of your module, such as the module docstring or the method docstring. These are defined inside their respective structures.

Now you have an idea of what happens when you call fputs() from the Python interpreter. The interpreter uses your module object as well as the module and method references to invoke the method. Finally, let’s take a look at how the interpreter handles the actual execution of your Python C extension module:

Once method_fputs() is invoked, the program executes the following steps:

  1. Parse the arguments you passed from the Python interpreter with PyArg_ParseTuple()
  2. Pass these arguments to fputs(), the C library function that forms the crux of your module
  3. Use PyLong_FromLong to return the value from fputs()

To see these same steps in code, take a look at method_fputs() again:

static PyObject *method_fputs(PyObject *self, PyObject *args) {

    char *str, *filename = NULL;

    int bytes_copied = -1;

    /* Parse arguments */

    if(!PyArg_ParseTuple(args, "ss", &str, &filename)) {

        return NULL;


    FILE *fp = fopen(filename, "w");

    bytes_copied = fputs(str, fp);


    return PyLong_FromLong(bytes_copied);


To recap, your method will parse the arguments passed to your module, send them on to fputs(), and return the results.

Packaging Your Python C Extension Module

Before you can import your new module, you first need to build it. You can do this by using the Python package distutils.

You’ll need a file called to install your application. For this tutorial, you’ll be focusing on the part specific to the Python C extension module.

A minimal file for your module should look like this:

from distutils.core import setup, Extension

def main():
          description="Python interface for the fputs C library function",
          author="<your name>",
          author_email="[email protected]",
          ext_modules=[Extension("fputs", ["fputsmodule.c"])])

if __name__ == "__main__":

The code block above shows the standard arguments that are passed to setup(). Take a closer look at the last positional argument, ext_modules. This takes a list of objects of the Extensions class. An object of the Extensions class describes a single C or C++ extension module in a setup script. Here, you pass two keyword arguments to its constructor, namely:

  • name is the name of the module.
  • [filename] is a list of paths to files with the source code, relative to the setup script.
[Remove ads](/account/join/)

Building Your Module

Now that you have your file, you can use it to build your Python C extension module. It’s strongly advised that you use a virtual environment to avoid conflicts with your Python environment.

Navigate to the directory containing and run the following command:

$ python3 install

This command will compile and install your Python C extension module in the current directory. If there are any errors or warnings, then your program will throw them now. Make sure you fix these before you try to import your module.

By default, the Python interpreter uses clang for compiling the C code. If you want to use gcc or any other C compiler for the job, then you need to set the CC environment variable accordingly, either inside the setup script or directly on the command line. For instance, you can tell the Python interpreter to use gcc to compile and build your module this way:

$ CC=gcc python3 install

However, the Python interpreter will automatically fall back to gcc if clang is not available.

Running Your Module

Now that everything is in place, it’s time to see your module in action! Once it’s successfully built, fire up the interpreter to test run your Python C extension module:

>>> import fputs
>>> fputs.__doc__
'Python interface for the fputs C library function'
>>> fputs.__name__
>>> # Write to an empty file named `write.txt`
>>> fputs.fputs("Real Python!", "write.txt")
>>> with open("write.txt", "r") as f:
>>>     print(
'Real Python!'

Your function performs as expected! You pass a string "Real Python!" and a file to write this string to, write.txt. The call to fputs() returns the number of bytes written to the file. You can verify this by printing the contents of the file.

Also recall how you passed certain arguments to the PyModuleDef and PyMethodDef structures. You can see from this output that Python has used these structures to assign things like the function name and docstring.

With that, you have a basic version of your module ready, but there’s a lot more that you can do! You can improve your module by adding things like custom exceptions and constants.

Raising Exceptions

Python exceptions are very different from C++ exceptions. If you want to raise Python exceptions from your C extension module, then you can use the Python API to do so. Some of the functions provided by the Python API for exception raising are as follows:

You can use any of these to raise an exception. However, which to use and when depends entirely on your requirements. The Python API has all the standard exceptions pre-defined as PyObject types.

Raising Exceptions From C Code

While you can’t raise exceptions in C, the Python API will allow you to raise exceptions from your Python C extension module. Let’s test this functionality by adding PyErr_SetString() to your code. This will raise an exception whenever the length of the string to be written is less than 10 characters:

static PyObject *method_fputs(PyObject *self, PyObject *args) {

    char *str, *filename = NULL;

    int bytes_copied = -1;

    /* Parse arguments */

    if(!PyArg_ParseTuple(args, "ss", &str, &fd)) {

        return NULL;


    if (strlen(str) < 10) {

        PyErr_SetString(PyExc_ValueError, "String length must be greater than 10");

        return NULL;


    fp = fopen(filename, "w");

    bytes_copied = fputs(str, fp);


    return PyLong_FromLong(bytes_copied);


Here, you check the length of the input string immediately after you parse the arguments and before you call fputs(). If the string passed by the user is shorter than 10 characters, then your program will raise a ValueError with a custom message. The program execution stops as soon as the exception occurs.

Note how method_fputs() returns NULL after raising the exception. This is because whenever you raise an exception using PyErr_*(), it automatically sets an internal entry in the exception table and returns it. The calling function is not required to subsequently set the entry again. For this reason, the calling function returns a value that indicates failure, usually NULL or -1. (This should also explain why there was a need to return NULL when you parse arguments in method_fputs() using PyArg_ParseTuple().)

Raising Custom Exceptions

You can also raise custom exceptions in your Python C extension module. However, things are a bit different. Previously, in PyMODINIT_FUNC, you were simply returning the instance returned by PyModule_Create and calling it a day. But for your custom exception to be accessible by the user of your module, you need to add your custom exception to your module instance before you return it:

static PyObject *StringTooShortError = NULL;

PyMODINIT_FUNC PyInit_fputs(void) {
    /* Assign module value */
    PyObject *module = PyModule_Create(&fputsmodule);

    /* Initialize new exception object */
    StringTooShortError = PyErr_NewException("fputs.StringTooShortError", NULL, NULL);

    /* Add exception object to your module */
    PyModule_AddObject(module, "StringTooShortError", StringTooShortError);

    return module;

As before, you start off by creating a module object. Then you create a new exception object using PyErr_NewException. This takes a string of the form module.classname as the name of the exception class that you wish to create. Choose something descriptive to make it easier for the user to interpret what has actually gone wrong.

Next, you add this to your module object using PyModule_AddObject. This takes your module object, the name of the new object being added, and the custom exception object itself as arguments. Finally, you return your module object.

Now that you’ve defined a custom exception for your module to raise, you need to update method_fputs() so that it raises the appropriate exception:

static PyObject *method_fputs(PyObject *self, PyObject *args) {

    char *str, *filename = NULL;

    int bytes_copied = -1;

    /* Parse arguments */

    if(!PyArg_ParseTuple(args, "ss", &str, &fd)) {

        return NULL;


    if (strlen(str) < 10) {

        /* Passing custom exception */

        PyErr_SetString(StringTooShortError, "String length must be greater than 10");

        return NULL;


    fp = fopen(filename, "w");

    bytes_copied = fputs(str, fp);


    return PyLong_FromLong(bytes_copied);


After building the module with the new changes, you can test that your custom exception is working as expected by trying to write a string that is less than 10 characters in length:

>>> import fputs
>>> # Custom exception
>>> fputs.fputs("RP!", fp.fileno())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
fputs.StringTooShortError: String length must be greater than 10

When you try to write a string with fewer than 10 characters, your custom exception is raised with a message explaining what went wrong.

Defining Constants

There are cases where you’ll want to use or define constants in your Python C extension module. This is quite similar to how you defined custom exceptions in the previous section. You can define a new constant and add it to your module instance using PyModule_AddIntConstant():

PyMODINIT_FUNC PyInit_fputs(void) {
    /* Assign module value */
    PyObject *module = PyModule_Create(&fputsmodule);

    /* Add int constant by name */
    PyModule_AddIntConstant(module, "FPUTS_FLAG", 64);

    /* Define int macro */
    #define FPUTS_MACRO 256

    /* Add macro to module */
    PyModule_AddIntMacro(module, FPUTS_MACRO);

    return module;

This Python API function takes the following arguments:

  • The instance of your module
  • The name of the constant
  • The value of the constant

You can do the same for macros using PyModule_AddIntMacro():

PyMODINIT_FUNC PyInit_fputs(void) {
    /* Assign module value */
    PyObject *module = PyModule_Create(&fputsmodule);

    /* Add int constant by name */
    PyModule_AddIntConstant(module, "FPUTS_FLAG", 64);

    /* Define int macro */
    #define FPUTS_MACRO 256

    /* Add macro to module */
    PyModule_AddIntMacro(module, FPUTS_MACRO);

    return module;

This function takes the following arguments:

  • The instance of your module
  • The name of the macro that has already been defined

Note: If you want to add string constants or macros to your module, then you can use PyModule_AddStringConstant() and PyModule_AddStringMacro(), respectively.

Open up the Python interpreter to see if your constants and macros are working as expected:

>>> import fputs
>>> # Constants
>>> fputs.FPUTS_FLAG
>>> fputs.FPUTS_MACRO

Here, you can see that the constants are accessible from within the Python interpreter.

[Remove ads](/account/join/)
Testing Your Module

You can test your Python C extension module just as you would any other Python module. This can be demonstrated by writing a small test function for pytest:

import fputs

def test_copy_data():
    content_to_copy = "Real Python!"
    bytes_copied = fputs.fputs(content_to_copy, 'test_write.txt')

    with open('test_write.txt', 'r') as f:
        content_copied =

    assert content_copied == content_to_copy

In the test script above, you use fputs.fputs() to write the string "Real Python!" to an empty file named test_write.txt. Then, you read in the contents of this file and use an assert statement to compare it to what you had originally written.

You can run this test suite to make sure your module is working as expected:

$ pytest -q                                                 [100%]
1 passed in 0.03 seconds
Considering Alternatives

In this tutorial, you’ve built an interface for a C library function to understand how to write Python C extension modules. However, there are times when all you need to do is invoke some system calls or a few C library functions, and you want to avoid the overhead of writing two different languages. In these cases, you can use Python libraries such as ctypes or cffi.

These are Foreign Function libraries for Python that provide access to C library functions and data types. Though the community itself is divided as to which library is best, both have their benefits and drawbacks. In other words, either would make a good choice for any given project, but there are a few things to keep in mind when you need to decide between the two:

  • The ctypes library comes included in the Python standard library. This is very important if you want to avoid external dependencies. It allows you to write wrappers for other languages in Python.

  • The cffi library is not yet included in the standard library. This might be a dealbreaker for your particular project. In general, it’s more Pythonic in nature, but it doesn’t handle preprocessing for you.

For more information on these libraries, check out Extending Python With C Libraries and the “ctypes” Module and Interfacing Python and C: The CFFI Module.

Note: Apart from ctypes and cffi, there are various other tools available. For instance, you can also use swig and boost::Py.


In this tutorial, you’ve learned how to write a Python interface in the C programming language using the Python API. You wrote a Python wrapper for the fputs() C library function. You also added custom exceptions and constants to your module before building and testing it.

The Python API provides a host of features for writing complex Python interfaces in the C programming language. At the same time, libraries such as cffi or ctypes can lower the amount of overhead involved in writing Python C extension modules. Make sure you weigh all the factors before making a decision!

Python String Formatting Best Practices

Python String Formatting Best Practices

In this tutorial, you’ll learn the four main approaches to string formatting in Python, as well as their strengths and weaknesses. You’ll also get a simple rule of thumb for how to pick the best general purpose string formatting approach in your own programs.

Remember the Zen of Python and how there should be “one obvious way to do something in Python”? You might scratch your head when you find out that there are four major ways to do string formatting in Python.

Table of Contents

  • #1 “Old Style” String Formatting (% Operator)
  • #2 “New Style” String Formatting (str.format)
  • #3 String Interpolation / f-Strings (Python 3.6+)
  • #4 Template Strings (Standard Library)
  • Which String Formatting Method Should You Use?
  • Key Takeaways

Let’s jump right in, as we’ve got a lot to cover. In order to have a simple toy example for experimentation, let’s assume you’ve got the following variables (or constants, really) to work with:

>>> errno = 50159747054
>>> name = 'Bob'

Based on these variables, you’d like to generate an output string containing a simple error message:

'Hey Bob, there is a 0xbadc0ffee error!'

That error could really spoil a dev’s Monday morning… But we’re here to discuss string formatting. So let’s get to work.

#1 “Old Style” String Formatting (% Operator)

Strings in Python have a unique built-in operation that can be accessed with the % operator. This lets you do simple positional formatting very easily. If you’ve ever worked with a printf-style function in C, you’ll recognize how this works instantly. Here’s a simple example:

>>> 'Hello, %s' % name
"Hello, Bob"

I’m using the %s format specifier here to tell Python where to substitute the value of name, represented as a string.

There are other format specifiers available that let you control the output format. For example, it’s possible to convert numbers to hexadecimal notation or add whitespace padding to generate nicely formatted tables and reports. (See Python Docs: “printf-style String Formatting”.)

Here, you can use the %x format specifier to convert an int value to a string and to represent it as a hexadecimal number:

>>> '%x' % errno

The “old style” string formatting syntax changes slightly if you want to make multiple substitutions in a single string. Because the % operator takes only one argument, you need to wrap the right-hand side in a tuple, like so:

>>> 'Hey %s, there is a 0x%x error!' % (name, errno)
'Hey Bob, there is a 0xbadc0ffee error!'

It’s also possible to refer to variable substitutions by name in your format string, if you pass a mapping to the % operator:

>>> 'Hey %(name)s, there is a 0x%(errno)x error!' % {
...     "name": name, "errno": errno }
'Hey Bob, there is a 0xbadc0ffee error!'

This makes your format strings easier to maintain and easier to modify in the future. You don’t have to worry about making sure the order you’re passing in the values matches up with the order in which the values are referenced in the format string. Of course, the downside is that this technique requires a little more typing.

I’m sure you’ve been wondering why this printf-style formatting is called “old style” string formatting. It was technically superseded by “new style” formatting in Python 3, which we’re going to talk about next.

#2 “New Style” String Formatting (str.format)

Python 3 introduced a new way to do string formatting that was also later back-ported to Python 2.7. This “new style” string formatting gets rid of the %-operator special syntax and makes the syntax for string formatting more regular. Formatting is now handled by calling .format() on a string object.

You can use format() to do simple positional formatting, just like you could with “old style” formatting:

>>> 'Hello, {}'.format(name)
'Hello, Bob'

Or, you can refer to your variable substitutions by name and use them in any order you want. This is quite a powerful feature as it allows for re-arranging the order of display without changing the arguments passed to format():

>>> 'Hey {name}, there is a 0x{errno:x} error!'.format(
...     name=name, errno=errno)
'Hey Bob, there is a 0xbadc0ffee error!'

This also shows that the syntax to format an int variable as a hexadecimal string has changed. Now you need to pass a format spec by adding a :x suffix. The format string syntax has become more powerful without complicating the simpler use cases. It pays off to read up on this string formatting mini-language in the Python documentation.

In Python 3, this “new style” string formatting is to be preferred over %-style formatting. While “old style” formatting has been de-emphasized, it has not been deprecated. It is still supported in the latest versions of Python. According to this discussion on the Python dev email list and this issue on the Python dev bug tracker, %-formatting is going to stick around for a long time to come.

Still, the official Python 3 documentation doesn’t exactly recommend “old style” formatting or speak too fondly of it:

“The formatting operations described here exhibit a variety of quirks that lead to a number of common errors (such as failing to display tuples and dictionaries correctly). Using the newer formatted string literals or the str.format() interface helps avoid these errors. These alternatives also provide more powerful, flexible and extensible approaches to formatting text.” (Source)

This is why I’d personally try to stick with str.format for new code moving forward. Starting with Python 3.6, there’s yet another way to format your strings. I’ll tell you all about it in the next section.

#3 String Interpolation / f-Strings (Python 3.6+)

Python 3.6 added a new string formatting approach called formatted string literals or “f-strings”. This new way of formatting strings lets you use embedded Python expressions inside string constants. Here’s a simple example to give you a feel for the feature:

>>> f'Hello, {name}!'
'Hello, Bob!'

As you can see, this prefixes the string constant with the letter “f“—hence the name “f-strings.” This new formatting syntax is powerful. Because you can embed arbitrary Python expressions, you can even do inline arithmetic with it. Check out this example:

>>> a = 5
>>> b = 10
>>> f'Five plus ten is {a + b} and not {2 * (a + b)}.'
'Five plus ten is 15 and not 30.'

Formatted string literals are a Python parser feature that converts f-strings into a series of string constants and expressions. They then get joined up to build the final string.

Imagine you had the following greet() function that contains an f-string:

>>> def greet(name, question):
...     return f"Hello, {name}! How's it {question}?"
>>> greet('Bob', 'going')
"Hello, Bob! How's it going?"

When you disassemble the function and inspect what’s going on behind the scenes, you’ll see that the f-string in the function gets transformed into something similar to the following:

>>> def greet(name, question):
...    return "Hello, " + name + "! How's it " + question + "?"

The real implementation is slightly faster than that because it uses the BUILD_STRING opcode as an optimization. But functionally they’re the same:

>>> import dis
>>> dis.dis(greet)
 2          0 LOAD_CONST              1 ('Hello, ')
             2 LOAD_FAST               0 (name)
             4 FORMAT_VALUE            0
             6 LOAD_CONST              2 ("! How's it ")
             8 LOAD_FAST               1 (question)
            10 FORMAT_VALUE            0
            12 LOAD_CONST              3 ('?')
            14 BUILD_STRING            5
            16 RETURN_VALUE

String literals also support the existing format string syntax of the str.format() method. That allows you to solve the same formatting problems we’ve discussed in the previous two sections:

>>> f"Hey {name}, there's a {errno:#x} error!"
"Hey Bob, there's a 0xbadc0ffee error!"

Python’s new formatted string literals are similar to JavaScript’s Template Literals added in ES2015. I think they’re quite a nice addition to Python, and I’ve already started using them in my day to day (Python 3) work.

#4 Template Strings (Standard Library)

Here’s one more tool for string formatting in Python: template strings. It’s a simpler and less powerful mechanism, but in some cases this might be exactly what you’re looking for.

Let’s take a look at a simple greeting example:

>>> from string import Template
>>> t = Template('Hey, $name!')
>>> t.substitute(name=name)
'Hey, Bob!'

You see here that we need to import the Template class from Python’s built-in string module. Template strings are not a core language feature but they’re supplied by the string module in the standard library.

Another difference is that template strings don’t allow format specifiers. So in order to get the previous error string example to work, you’ll need to manually transform the int error number into a hex-string:

>>> templ_string = 'Hey $name, there is a $error error!'
>>> Template(templ_string).substitute(
...     name=name, error=hex(errno))
'Hey Bob, there is a 0xbadc0ffee error!'

That worked great.

So when should you use template strings in your Python programs? In my opinion, the best time to use template strings is when you’re handling formatted strings generated by users of your program. Due to their reduced complexity, template strings are a safer choice.

The more complex formatting mini-languages of the other string formatting techniques might introduce security vulnerabilities to your programs. For example, it’s possible for format strings to access arbitrary variables in your program.

That means, if a malicious user can supply a format string, they can potentially leak secret keys and other sensitive information! Here’s a simple proof of concept of how this attack might be used against your code:

>>> # This is our super secret key:
>>> SECRET = 'this-is-a-secret'

>>> class Error:
...      def init(self):
...          pass

>>> # A malicious user can craft a format string that
>>> # can read data from the global namespace:
>>> user_input = '{error.init.globals[SECRET]}'

>>> # This allows them to exfiltrate sensitive information,
>>> # like the secret key:
>>> err = Error()
>>> user_input.format(error=err)

See how a hypothetical attacker was able to extract our secret string by accessing the globals dictionary from a malicious format string? Scary, huh? Template strings close this attack vector. This makes them a safer choice if you’re handling format strings generated from user input:

>>> user_input = '${error.init.globals[SECRET]}'
>>> Template(user_input).substitute(error=err)
"Invalid placeholder in string: line 1, col 1"
Which String Formatting Method Should You Use?

I totally get that having so much choice for how to format your strings in Python can feel very confusing. This is an excellent cue to bust out this handy flowchart infographic I’ve put together for you:

Python String Formatting Rule of Thumb

Key Takeaways
  • Perhaps surprisingly, there’s more than one way to handle string formatting in Python.
  • Each method has its individual pros and cons. Your use case will influence which method you should use.
  • If you’re having trouble deciding which string formatting method to use, try our Python String Formatting Rule of Thumb.

Thanks for reading

If you liked this post, share it with all of your programming buddies!

Follow us on Facebook | Twitter

Further reading

Python Programming Tutorial - Full Course for Beginners

Python 3's f-Strings: Usage Guide

Python 3's f-Strings: A guide to String Formatting Syntax

How to build a JSON API with Python

How to build a JSON API with Python

The JSON API specification is a powerful way for enabling communication between client and server. It specifies the structure of the requests and responses sent between the two, using the JSON format. The [JSON API...

The JSON API specification is a powerful way for enabling communication between client and server. It specifies the structure of the requests and responses sent between the two, using the JSON format.

The JSON API specification is a powerful way for enabling communication between client and server. It specifies the structure of the requests and responses sent between the two, using the JSON format.

As a data format, JSON has the advantages of being lightweight and readable. This makes it very easy to work with quickly and productively. The specification is designed to minimise the number of requests and the amount of data that needs sending between client and server.

Here, you can learn how to create a basic JSON API using Python and Flask. Then, the rest of the article will show you how to try out some of the features the JSON API specification has to offer.

Flask is a Python library that provides a 'micro-framework' for web development. It is great for rapid development as it comes with a simple-yet-extensible core functionality.

A really basic example of how to send a JSON-like response using Flask is shown below:

from flask import Flask

app = Flask(__name__)

def example():
   return '{"name":"Bob"}'

if __name__ == '__main__':

This article will use two add-ons for Flask:

The big picture

The end goal is to create an API that allows client-side interaction with an underlying database. There will be a couple of layers between the database and the client - a data abstraction layer and a resource manager layer.

Here's an overview of the steps involved:

  1. Define a database using Flask-SQLAlchemy
  2. Create a data abstraction with Marshmallow-JSONAPI
  3. Create resource managers with Flask-REST-JSONAPI
  4. Create URL endpoints and start the server with Flask

This example will use a simple schema describing modern artists and their relationships to different artworks.

Install everything

Before getting started, you'll need to set up the project. This involves creating a workspace and virtual environment, installing the modules required, and creating the main Python and database files for the project.

From the command line create a new directory and navigate inside.

$ mkdir flask-jsonapi-demo
$ cd flask-jsonapi-demo/

It is good practice to create virtual environments for each of your Python projects. You can skip this step, but it is strongly recommended.

$ python -m venv .venv
$ source .venv/bin/activate

Once your virtual environment has been created and activated, you can install the modules needed for this project.

$ pip install flask-rest-jsonapi flask-sqlalchemy

Everything you'll need will be installed as the requirements for these two extensions. This includes Flask itself, and SQLAlchemy.

The next step is to create a Python file and database for the project.

$ touch artists.db

Create the database schema

Here, you will start modifying to define and create the database schema for the project.

Open in your preferred text editor. Begin by importing some modules. For clarity, modules will be imported as you go.

Next, create an object called app as an instance of the Flask class.

After that, use SQLAlchemy to connect to the database file you created. The final step is to define and create a table called artists.

from flask import Flask
from flask_sqlalchemy import SQLAlchemy

# Create a new Flask application
app = Flask(__name__)

# Set up SQLAlchemy
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////artists.db'
db = SQLAlchemy(app)

# Define a class for the Artist table
class Artist(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    name = db.Column(db.String)
    birth_year = db.Column(db.Integer)
    genre = db.Column(db.String)

# Create the table

Creating an abstraction layer

The next step uses the Marshmallow-JSONAPI module to create a logical data abstraction over the tables just defined.

The reason to create this abstraction layer is simple. It gives you more control over how your underlying data is exposed via the API. Think of this layer as a lens through which the API client can view the underlying data clearly, and only the bits they need to see.

In the code below, the data abstraction layer is defined as a class which inherits from Marshmallow-JSONAPI's Schema class. It will provide access via the API to both single records and multiple records from the artists table.

Inside this block, the Meta class defines some metadata. Specifically, the name of the URL endpoint for interacting with single records will be artist_one, where each artist will be identified by a URL parameter <id>. The name of the endpoint for interacting with many records will be artist_many.

The remaining attributes defined relate to the columns in the artists table. Here, you can control further how each is exposed via the API.

For example, when making POST requests to add new artists to the database, you can make sure the name field is mandatory by setting required=True.

And if for any reason you didn't want the birth_year field to be returned when making GET requests, you can specify so by setting load_only=True.

from marshmallow_jsonapi.flask import Schema
from marshmallow_jsonapi import fields

# Create data abstraction layer
class ArtistSchema(Schema):
    class Meta:
        type_ = 'artist'
        self_view = 'artist_one'
        self_view_kwargs = {'id': '<id>'}
        self_view_many = 'artist_many'

    id = fields.Integer()
    name = fields.Str(required=True)
    birth_year = fields.Integer(load_only=True)
    genre = fields.Str()

Create resource managers and URL endpoints

The final piece of the puzzle is to create a resource manager and corresponding endpoint for each of the routes /artists and /artists/id.

Each resource manager is defined as a class that inherits from the Flask-REST-JSONAPI classes ResourceList and ResourceDetail.

Here they take two attributes. schema is used to indicate the data abstraction layer the resource manager uses, and data_layer indicates the session and data model that will be used for the data layer.

Next, define api as an instance of Flask-REST-JSONAPI's Api class, and create the routes for the API with api.route(). This method takes three arguments - the data abstraction layer class, the endpoint name, and the URL path.

The last step is to write a main loop to launch the app in debug mode when the script is run directly. Debug mode is great for development, but it is not suitable for running in production.

# Create resource managers and endpoints

from flask_rest_jsonapi import Api, ResourceDetail, ResourceList

class ArtistMany(ResourceList):
    schema = ArtistSchema
    data_layer = {'session': db.session,
                  'model': Artist}

class ArtistOne(ResourceDetail):
    schema = ArtistSchema
    data_layer = {'session': db.session,
                  'model': Artist}

api = Api(app)
api.route(ArtistMany, 'artist_many', '/artists')
api.route(ArtistOne, 'artist_one', '/artists/<int:id>')

# main loop to run app in debug mode
if __name__ == '__main__':

Make GET and POST requests

Now you can start using the API to make HTTP requests. This could be from a web browser, or from a command line tool like curl, or from within another program (e.g., a Python script using the Requests library).

To launch the server, run the script with:

$ python

In your browser, navigate to http://localhost:5000/artists.  You will see a JSON output of all the records in the database so far. Except, there are none.

To start adding records to the database, you can make a POST request. One way of doing this is from the command line using curl. Alternatively, you could use a tool like Insomnia, or perhaps code up a simple HTML user interface that posts data using a form.

With curl, from the command line:

curl -i -X POST -H 'Content-Type: application/json' -d '{"data":{"type":"artist", "attributes":{"name":"Salvador Dali", "birth_year":1904, "genre":"Surrealism"}}}' http://localhost:5000/artists

Now if you navigate to http://localhost:5000/artists, you will see the record you just added. If you were to add more records, they would all show here as well, as this URL path calls the artists_many endpoint.

To view just a single artist by their id number, you can navigate to the relevant URL. For example, to see the first artist, try http://localhost:5000/artists/1.

Filtering and sorting

One of the neat features of the JSON API specification is the ability to return the response in more useful ways by defining some parameters in the URL. For instance, you can sort the results according to a chosen field, or filter based on some criteria.

Flask-REST-JSONAPI comes with this built in.

To sort artists in order of birth year, just navigate to http://localhost:5000/artists?sort=birth_year. In a web application, this would save you from needing to sort results on the client side, which could be costly in terms of performance and therefore impact the user experience.

Filtering is also easy. You append to the URL the criteria you wish to filter on, contained in square brackets. There are three pieces of information to include:

  • "name" - the field you are filtering by (e.g., birth_year)
  • "op" - the filter operation ("equal to", "greater than", "less than" etc.)
  • "val" - the value to filter against (e.g., 1900)

For example, the URL below retrieves artists whose birth year is greater than 1900:


This functionality makes it much easier to retrieve only relevant information when calling the API. This is valuable for improving performance, especially when retrieving potentially large volumes of data over a slow connection.


Another feature of the JSON API specification that aids performance is pagination. This is when large responses are sent over several "pages", rather than all in one go. You can control the page size and the number of the page you request in the URL.

So, for example, you could receive 100 results over 10 pages instead of loading all 100 in one go. The first page would contain results 1-10, the second page would contain results 11-20, and so on.

To specify the number of results you want to receive per page, you can add the parameter ?page[size]=X to the URL, where X is the number of results. Flask-REST-JSONAPI uses 30 as the default page size.

To request a given page number, you can add the parameter ?page[number]=X, where is the page number. You can combine both parameters as shown below:


This URL sets the page size to two results per page, and asks for the second page of results. This would return the third and fourth results from the overall response.


Almost always, data in one table will be related to data stored in another. For instance, if you have a table of artists, chances are you might also want a table of artworks. Each artwork is related to the artist who created it.

The JSON API specification allows you to work with relational data easily, and the Flask-REST-JSONAPI lets you take advantage of this. Here, this will be demonstrated by adding an artworks table to the database, and including relationships between artist and artwork.

To implement the artworks example, it will be necessary to make a few changes to the code in

First, make a couple of extra imports, then create a new table which relates each artwork to an artist:

from marshmallow_jsonapi.flask import Relationship
from flask_rest_jsonapi import ResourceRelationship

# Define the Artwork table
class Artwork(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    title = db.Column(db.String)
    artist_id = db.Column(db.Integer, 
    artist = db.relationship('Artist',

Next, rewrite the abstraction layer:

# Create data abstraction 
class ArtistSchema(Schema):
    class Meta:
        type_ = 'artist'
        self_view = 'artist_one'
        self_view_kwargs = {'id': '<id>'}
        self_view_many = 'artist_many'

    id = fields.Integer()
    name = fields.Str(required=True)
    birth_year = fields.Integer(load_only=True)
    genre = fields.Str()
    artworks = Relationship(self_view = 'artist_artworks',
        self_view_kwargs = {'id': '<id>'},
        related_view = 'artwork_many',
        many = True,
        schema = 'ArtworkSchema',
        type_ = 'artwork')

class ArtworkSchema(Schema):
    class Meta:
        type_ = 'artwork'
        self_view = 'artwork_one'
        self_view_kwargs = {'id': '<id>'}
        self_view_many = 'artwork_many'

    id = fields.Integer()
    title = fields.Str(required=True)
    artist_id = fields.Integer(required=True)

This defines an abstraction layer for the artwork table, and adds a relationship between artist and artwork to the ArtistSchema class.

Next, define new resource managers for accessing artworks many at once and one at a time, and also for accessing the relationships between artist and artwork.

class ArtworkMany(ResourceList):
    schema = ArtworkSchema
    data_layer = {'session': db.session,
                  'model': Artwork}

class ArtworkOne(ResourceDetail):
    schema = ArtworkSchema
    data_layer = {'session': db.session,
                  'model': Artwork}

class ArtistArtwork(ResourceRelationship):
    schema = ArtistSchema
    data_layer = {'session': db.session,
                  'model': Artist}

Finally, add some new endpoints:

api.route(ArtworkOne, 'artwork_one', '/artworks/<int:id>')
api.route(ArtworkMany, 'artwork_many', '/artworks')
api.route(ArtistArtwork, 'artist_artworks',

Run and trying posting some data from the command line via curl:

curl -i -X POST -H 'Content-Type: application/json' -d '{"data":{"type":"artwork", "attributes":{"title":"The Persistance of Memory", "artist_id":1}}}' http://localhost:5000/artworks

This will create an artwork related to the artist with id=1.

In the browser, navigate to http://localhost:5000/artists/1/relationships/artworks. This should show the artworks related to the artist with id=1. This saves you from writing a more complex URL with parameters to filter artworks by their artist_id field. You can quickly list all the relationships between a given artist and their artworks.

Another feature is the ability to include related results in the response to calling the artists_one endpoint:


This will return the usual response for the artists endpoint, and also results for each of that artist's artworks.

Sparse Fields

One last feature worth mentioning - sparse fields. When working with large data resources with many complex relationships, the response sizes can blow up real fast. It is helpful to only retrieve the fields you are interested in.

The JSON API specification lets you do this by adding a fields parameter to the URL. For example URL below gets the response for a given artist and their related artworks. However, instead of returning all the fields for the given artwork, it returns only the title.


This is again very helpful for improving performance, especially over slow connections. As a general rule, you should only make requests to and from the server with the minimal amount of data required.

Final remarks

The JSON API specification is a very useful framework for sending data between server and client in a clean, flexible format. This article has provided an overview of what you can do with it, with a worked example in Python using the Flask-REST-JSONAPI library.

So what will you do next? There are many possibilities. The example in this article has been a simple proof-of-concept, with just two tables and a single relationship between them. You can develop an application as sophisticated as you like, and create a powerful API to interact with it using all the tools provided here.

Thanks for reading, and keep coding in Python!