Getting Started With Testing in Python

Getting Started With Testing in Python

In this tutorial, you’ll learn how to create a basic test, execute it, and find the bugs before your users do! You’ll learn about the tools available to write and execute tests, check your application’s performance, and even look for security issues.

This tutorial is for anyone who has written a fantastic application in Python but hasn’t yet written any tests.

Testing in Python is a huge topic and can come with a lot of complexity, but it doesn’t need to be hard. You can get started creating simple tests for your application in a few easy steps and then build on it from there.

Table of Contents

  • Testing Your Code
    • Automated vs. Manual Testing
    • Unit Tests vs. Integration Tests
    • Choosing a Test Runner
  • Writing Your First Test
    • Where to Write the Test
    • How to Structure a Simple Test
    • How to Write Assertions
    • Side Effects
  • Executing Your First Test
    • Executing Test Runners
    • Understanding Test Output
    • Running Your Tests From PyCharm
    • Running Your Tests From Visual Studio Code
  • Testing for Web Frameworks Like Django and Flask
    • Why They’re Different From Other Applications
    • How to Use the Django Test Runner
    • How to Use unittest and Flask
  • More Advanced Testing Scenarios
    • Handling Expected Failures
    • Isolating Behaviors in Your Application
    • Writing Integration Tests
    • Testing Data-Driven Applications
  • Testing in Multiple Environments
    • Installing Tox
    • Configuring Tox for Your Dependencies
    • Executing Tox
  • Automating the Execution of Your Tests
  • What’s Next
    • Introducing Linters Into Your Application
    • Keeping Your Test Code Clean
    • Testing for Performance Degradation Between Changes
    • Testing for Security Flaws in Your Application
  • Conclusion
Testing Your Code

There are many ways to test your code. In this tutorial, you’ll learn the techniques from the most basic steps and work towards advanced methods.

Automated vs. Manual Testing

The good news is, you’ve probably already created a test without realizing it. Remember when you ran your application and used it for the first time? Did you check the features and experiment using them? That’s known as exploratory testing and is a form of manual testing.

Exploratory testing is a form of testing that is done without a plan. In an exploratory test, you’re just exploring the application.

To have a complete set of manual tests, all you need to do is make a list of all the features your application has, the different types of input it can accept, and the expected results. Now, every time you make a change to your code, you need to go through every single item on that list and check it.

That doesn’t sound like much fun, does it?

This is where automated testing comes in. Automated testing is the execution of your test plan (the parts of your application you want to test, the order in which you want to test them, and the expected responses) by a script instead of a human. Python already comes with a set of tools and libraries to help you create automated tests for your application. We’ll explore those tools and libraries in this tutorial.

Unit Tests vs. Integration Tests

The world of testing has no shortage of terminology, and now that you know the difference between automated and manual testing, it’s time to go a level deeper.

Think of how you might test the lights on a car. You would turn on the lights (known as the test step) and go outside the car or ask a friend to check that the lights are on (known as the test assertion). Testing multiple components is known as integration testing.

Think of all the things that need to work correctly in order for a simple task to give the right result. These components are like the parts to your application, all of those classes, functions, and modules you’ve written.

A major challenge with integration testing is when an integration test doesn’t give the right result. It’s very hard to diagnose the issue without being able to isolate which part of the system is failing. If the lights didn’t turn on, then maybe the bulbs are broken. Is the battery dead? What about the alternator? Is the car’s computer failing?

If you have a fancy modern car, it will tell you when your light bulbs have gone. It does this using a form of unit test.

A unit test is a smaller test, one that checks that a single component operates in the right way. A unit test helps you to isolate what is broken in your application and fix it faster.

You have just seen two types of tests:

  1. An integration test checks that components in your application operate with each other.
  2. A unit test checks a small component in your application.

You can write both integration tests and unit tests in Python. To write a unit test for the built-in function sum(), you would check the output of sum() against a known output.

For example, here’s how you check that the sum() of the numbers (1, 2, 3) equals 6:

>>> assert sum([1, 2, 3]) == 6, "Should be 6"

This will not output anything on the REPL because the values are correct.

If the result from sum() is incorrect, this will fail with an AssertionError and the message "Should be 6". Try an assertion statement again with the wrong values to see an AssertionError:

>>> assert sum([1, 1, 1]) == 6, "Should be 6"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AssertionError: Should be 6

In the REPL, you are seeing the raised AssertionError because the result of sum() does not match 6.

Instead of testing on the REPL, you’ll want to put this into a new Python file called test_sum.py and execute it again:

def test_sum():
    assert sum([1, 2, 3]) == 6, "Should be 6"

if __name__ == "__main__":
    test_sum()
    print("Everything passed")

Now you have written a test case, an assertion, and an entry point (the command line). You can now execute this at the command line:

$ python test_sum.py
Everything passed

You can see the successful result, Everything passed.

In Python, sum() accepts any iterable as its first argument. You tested with a list. Now test with a tuple as well. Create a new file called test_sum_2.py with the following code:

def test_sum():
    assert sum([1, 2, 3]) == 6, "Should be 6"

def test_sum_tuple():
    assert sum((1, 2, 2)) == 6, "Should be 6"

if __name__ == "__main__":
    test_sum()
    test_sum_tuple()
    print("Everything passed")

When you execute test_sum_2.py, the script will give an error because the sum() of (1, 2, 2) is 5, not 6. The result of the script gives you the error message, the line of code, and the traceback:

$ python test_sum_2.py
Traceback (most recent call last):
  File "test_sum_2.py", line 9, in <module>
    test_sum_tuple()
  File "test_sum_2.py", line 5, in test_sum_tuple
    assert sum((1, 2, 2)) == 6, "Should be 6"
AssertionError: Should be 6

Here you can see how a mistake in your code gives an error on the console with some information on where the error was and what the expected result was.

Writing tests in this way is okay for a simple check, but what if more than one fails? This is where test runners come in. The test runner is a special application designed for running tests, checking the output, and giving you tools for debugging and diagnosing tests and applications.

Choosing a Test Runner

There are many test runners available for Python. The one built into the Python standard library is called unittest. In this tutorial, you will be using unittest test cases and the unittest test runner. The principles of unittest are easily portable to other frameworks. The three most popular test runners are:

  • unittest
  • nose or nose2
  • pytest

Choosing the best test runner for your requirements and level of experience is important.

unittest

unittest has been built into the Python standard library since version 2.1. You’ll probably see it in commercial Python applications and open-source projects.

unittest contains both a testing framework and a test runner. unittest has some important requirements for writing and executing tests.

unittest requires that:

  • You put your tests into classes as methods
  • You use a series of special assertion methods in the unittest.TestCase class instead of the built-in assert statement

To convert the earlier example to a unittest test case, you would have to:

  1. Import unittest from the standard library
  2. Create a class called TestSum that inherits from the TestCase class
  3. Convert the test functions into methods by adding self as the first argument
  4. Change the assertions to use the self.assertEqual() method on the TestCase class
  5. Change the command-line entry point to call unittest.main()

Follow those steps by creating a new file test_sum_unittest.py with the following code:

import unittest


class TestSum(unittest.TestCase):

    def test_sum(self):
        self.assertEqual(sum([1, 2, 3]), 6, "Should be 6")

    def test_sum_tuple(self):
        self.assertEqual(sum((1, 2, 2)), 6, "Should be 6")

if __name__ == '__main__':
    unittest.main()

If you execute this at the command line, you’ll see one success (indicated with .) and one failure (indicated with F):

$ python test_sum_unittest.py
.F
======================================================================
FAIL: test_sum_tuple (__main__.TestSum)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_sum_unittest.py", line 9, in test_sum_tuple
    self.assertEqual(sum((1, 2, 2)), 6, "Should be 6")
AssertionError: Should be 6

----------------------------------------------------------------------
Ran 2 tests in 0.001s

FAILED (failures=1)

You have just executed two tests using the unittest test runner.

Note: Be careful if you’re writing test cases that need to execute in both Python 2 and 3. In Python 2.7 and below, unittest is called unittest2. If you simply import from unittest, you will get different versions with different features between Python 2 and 3.

For more information on unittest, you can explore the unittest Documentation.

nose

You may find that over time, as you write hundreds or even thousands of tests for your application, it becomes increasingly hard to understand and use the output from unittest.

nose is compatible with any tests written using the unittest framework and can be used as a drop-in replacement for the unittest test runner. The development of nose as an open-source application fell behind, and a fork called nose2 was created. If you’re starting from scratch, it is recommended that you use nose2 instead of nose.

To get started with nose2, install nose2 from PyPI and execute it on the command line. nose2 will try to discover all test scripts named test*.py and test cases inheriting from unittest.TestCase in your current directory:

$ pip install nose2
$ python -m nose2
.F
======================================================================
FAIL: test_sum_tuple (__main__.TestSum)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_sum_unittest.py", line 9, in test_sum_tuple
    self.assertEqual(sum((1, 2, 2)), 6, "Should be 6")
AssertionError: Should be 6

----------------------------------------------------------------------
Ran 2 tests in 0.001s

FAILED (failures=1)

You have just executed the test you created in test_sum_unittest.py from the nose2 test runner. nose2 offers many command-line flags for filtering the tests that you execute. For more information, you can explore the Nose 2 documentation.

pytest

pytest supports execution of unittest test cases. The real advantage of pytest comes by writing pytest test cases. pytest test cases are a series of functions in a Python file starting with the name test_.

pytest has some other great features:

  • Support for the built-in assert statement instead of using special self.assert*() methods
  • Support for filtering for test cases
  • Ability to rerun from the last failing test
  • An ecosystem of hundreds of plugins to extend the functionality

Writing the TestSum test case example for pytest would look like this:

def test_sum():
    assert sum([1, 2, 3]) == 6, "Should be 6"

def test_sum_tuple():
    assert sum((1, 2, 2)) == 6, "Should be 6"

You have dropped the TestCase, any use of classes, and the command-line entry point.

More information can be found at the Pytest Documentation Website.

Writing Your First Test

Let’s bring together what you’ve learned so far and, instead of testing the built-in sum() function, test a simple implementation of the same requirement.

Create a new project folder and, inside that, create a new folder called my_sum. Inside my_sum, create an empty file called __init__.py. Creating the __init__.py file means that the my_sum folder can be imported as a module from the parent directory.

Your project folder should look like this:

project/
│
└── my_sum/
    └── __init__.py

Open up my_sum/__init__.py and create a new function called sum(), which takes an iterable (a list, tuple, or set) and adds the values together:

def sum(arg):
    total = 0
    for val in arg:
        total += val
    return total

This code example creates a variable called total, iterates over all the values in arg, and adds them to total. It then returns the result once the iterable has been exhausted.

Where to Write the Test

To get started writing tests, you can simply create a file called test.py, which will contain your first test case. Because the file will need to be able to import your application to be able to test it, you want to place test.py above the package folder, so your directory tree will look something like this:

project/
│
├── my_sum/
│   └── __init__.py
|
└── test.py

You’ll find that, as you add more and more tests, your single file will become cluttered and hard to maintain, so you can create a folder called tests/ and split the tests into multiple files. It is convention to ensure each file starts with test_ so all test runners will assume that Python file contains tests to be executed. Some very large projects split tests into more subdirectories based on their purpose or usage.

Note: What if your application is a single script?

You can import any attributes of the script, such as classes, functions, and variables by using the built-in __import__() function. Instead of from my_sum import sum, you can write the following:

target = __import__("my_sum.py")
sum = target.sum

The benefit of using __import__() is that you don’t have to turn your project folder into a package, and you can specify the file name. This is also useful if your filename collides with any standard library packages. For example, math.py would collide with the math module.

How to Structure a Simple Test

Before you dive into writing tests, you’ll want to first make a couple of decisions:

  1. What do you want to test?
  2. Are you writing a unit test or an integration test?

Then the structure of a test should loosely follow this workflow:

  1. Create your inputs
  2. Execute the code being tested, capturing the output
  3. Compare the output with an expected result

For this application, you’re testing sum(). There are many behaviors in sum() you could check, such as:

  • Can it sum a list of whole numbers (integers)?
  • Can it sum a tuple or set?
  • Can it sum a list of floats?
  • What happens when you provide it with a bad value, such as a single integer or a string?
  • What happens when one of the values is negative?

The most simple test would be a list of integers. Create a file, test.py with the following Python code:

import unittest

from my_sum import sum


class TestSum(unittest.TestCase):
    def test_list_int(self):
        """
        Test that it can sum a list of integers
        """
        data = [1, 2, 3]
        result = sum(data)
        self.assertEqual(result, 6)

if __name__ == '__main__':
    unittest.main()

This code example:

  1. Imports sum() from the my_sum package you created

  2. Defines a new test case class called TestSum, which inherits from unittest.TestCase

  3. Defines a test method, .test_list_int(), to test a list of integers. The method .test_list_int() will:

    • Declare a variable data with a list of numbers (1, 2, 3)
    • Assign the result of my_sum.sum(data) to a result variable
    • Assert that the value of result equals 6 by using the .assertEqual() method on the unittest.TestCase class
  4. Defines a command-line entry point, which runs the unittest test-runner .main()

If you’re unsure what self is or how .assertEqual() is defined, you can brush up on your object-oriented programming with Python 3 Object-Oriented Programming.

How to Write Assertions

The last step of writing a test is to validate the output against a known response. This is known as an assertion. There are some general best practices around how to write assertions:

  • Make sure tests are repeatable and run your test multiple times to make sure it gives the same result every time
  • Try and assert results that relate to your input data, such as checking that the result is the actual sum of values in the sum() example

unittest comes with lots of methods to assert on the values, types, and existence of variables. Here are some of the most commonly used methods:

.assertIs(), .assertIsNone(), .assertIn(), and .assertIsInstance() all have opposite methods, named .assertIsNot(), and so forth.

Side Effects

When you’re writing tests, it’s often not as simple as looking at the return value of a function. Often, executing a piece of code will alter other things in the environment, such as the attribute of a class, a file on the filesystem, or a value in a database. These are known as side effects and are an important part of testing. Decide if the side effect is being tested before including it in your list of assertions.

If you find that the unit of code you want to test has lots of side effects, you might be breaking the Single Responsibility Principle. Breaking the Single Responsibility Principle means the piece of code is doing too many things and would be better off being refactored. Following the Single Responsibility Principle is a great way to design code that it is easy to write repeatable and simple unit tests for, and ultimately, reliable applications.

Executing Your First Test

Now that you’ve created the first test, you want to execute it. Sure, you know it’s going to pass, but before you create more complex tests, you should check that you can execute the tests successfully.

Executing Test Runners

The Python application that executes your test code, checks the assertions, and gives you test results in your console is called the test runner.

At the bottom of test.py, you added this small snippet of code:

if __name__ == '__main__':
    unittest.main()

This is a command line entry point. It means that if you execute the script alone by running python test.py at the command line, it will call unittest.main(). This executes the test runner by discovering all classes in this file that inherit from unittest.TestCase.

This is one of many ways to execute the unittest test runner. When you have a single test file named test.py, calling python test.py is a great way to get started.

Another way is using the unittest command line. Try this:

$ python -m unittest test

This will execute the same test module (called test) via the command line.

You can provide additional options to change the output. One of those is -v for verbose. Try that next:

$ python -m unittest -v test
test_list_int (test.TestSum) ... ok

----------------------------------------------------------------------
Ran 1 tests in 0.000s

This executed the one test inside test.py and printed the results to the console. Verbose mode listed the names of the tests it executed first, along with the result of each test.

Instead of providing the name of a module containing tests, you can request an auto-discovery using the following:

$ python -m unittest discover

This will search the current directory for any files named test*.py and attempt to test them.

Once you have multiple test files, as long as you follow the test*.py naming pattern, you can provide the name of the directory instead by using the -s flag and the name of the directory:

$ python -m unittest discover -s tests

unittest will run all tests in a single test plan and give you the results.

Lastly, if your source code is not in the directory root and contained in a subdirectory, for example in a folder called src/, you can tell unittest where to execute the tests so that it can import the modules correctly with the -t flag:

$ python -m unittest discover -s tests -t src

unittest will change to the src/ directory, scan for all test*.py files inside the the tests directory, and execute them.

Understanding Test Output

That was a very simple example where everything passes, so now you’re going to try a failing test and interpret the output.

sum() should be able to accept other lists of numeric types, like fractions.

At the top of the test.py file, add an import statement to import the Fraction type from the fractions module in the standard library:

from fractions import Fraction

Now add a test with an assertion expecting the incorrect value, in this case expecting the sum of 1/4, 1/4, and 2/5 to be 1:

import unittest

from my_sum import sum


class TestSum(unittest.TestCase):
    def test_list_int(self):
        """
        Test that it can sum a list of integers
        """
        data = [1, 2, 3]
        result = sum(data)
        self.assertEqual(result, 6)

    def test_list_fraction(self):
        """
        Test that it can sum a list of fractions
        """
        data = [Fraction(1, 4), Fraction(1, 4), Fraction(2, 5)]
        result = sum(data)
        self.assertEqual(result, 1)

if __name__ == '__main__':
    unittest.main()

If you execute the tests again with python -m unittest test, you should see the following output:

$ python -m unittest test
F.
======================================================================
FAIL: test_list_fraction (test.TestSum)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test.py", line 21, in test_list_fraction
    self.assertEqual(result, 1)
AssertionError: Fraction(9, 10) != 1

----------------------------------------------------------------------
Ran 2 tests in 0.001s

FAILED (failures=1)

In the output, you’ll see the following information:

  1. The first line shows the execution results of all the tests, one failed (F) and one passed (.).

  2. The FAIL entry shows some details about the failed test:

    • The test method name (test_list_fraction)
    • The test module (test) and the test case (TestSum)
    • A traceback to the failing line
    • The details of the assertion with the expected result (1) and the actual result (Fraction(9, 10))

Remember, you can add extra information to the test output by adding the -v flag to the python -m unittest command.

Running Your Tests From PyCharm

If you’re using the PyCharm IDE, you can run unittest or pytest by following these steps:

  1. In the Project tool window, select the tests directory.
  2. On the context menu, choose the run command for unittest. For example, choose Run ‘Unittests in my Tests…’.

This will execute unittest in a test window and give you the results within PyCharm:

More information is available on the PyCharm Website.

Running Your Tests From Visual Studio Code

If you’re using the Microsoft Visual Studio Code IDE, support for unittest, nose, and pytest execution is built into the Python plugin.

If you have the Python plugin installed, you can set up the configuration of your tests by opening the Command Palette with Ctrl+Shift+P and typing “Python test”. You will see a range of options:

Choose Debug All Unit Tests, and VSCode will then raise a prompt to configure the test framework. Click on the cog to select the test runner (unittest) and the home directory (.).

Once this is set up, you will see the status of your tests at the bottom of the window, and you can quickly access the test logs and run the tests again by clicking on these icons:

This shows the tests are executing, but some of them are failing.

Testing for Web Frameworks Like Django and Flask

If you’re writing tests for a web application using one of the popular frameworks like Django or Flask, there are some important differences in the way you write and run the tests.

Why They’re Different From Other Applications

Think of all the code you’re going to be testing in a web application. The routes, views, and models all require lots of imports and knowledge about the frameworks being used.

This is similar to the car test at the beginning of the tutorial: you have to start up the car’s computer before you can run a simple test like checking the lights.

Django and Flask both make this easy for you by providing a test framework based on unittest. You can continue writing tests in the way you’ve been learning but execute them slightly differently.

How to Use the Django Test Runner

The Django startapp template will have created a tests.py file inside your application directory. If you don’t have that already, you can create it with the following contents:

from django.test import TestCase

class MyTestCase(TestCase):
    # Your test methods

The major difference with the examples so far is that you need to inherit from the django.test.TestCase instead of unittest.TestCase. These classes have the same API, but the Django TestCase class sets up all the required state to test.

To execute your test suite, instead of using unittest at the command line, you use manage.py test:

$ python manage.py test

If you want multiple test files, replace tests.py with a folder called tests, insert an empty file inside called __init__.py, and create your test_*.py files. Django will discover and execute these.

More information is available at the Django Documentation Website.

How to Use unittest and Flask

Flask requires that the app be imported and then set in test mode. You can instantiate a test client and use the test client to make requests to any routes in your application.

All of the test client instantiation is done in the setUp method of your test case. In the following example, my_app is the name of the application. Don’t worry if you don’t know what setUp does. You’ll learn about that in the More Advanced Testing Scenarios section.

The code within your test file should look like this:

import my_app
import unittest


class MyTestCase(unittest.TestCase):

    def setUp(self):
        my_app.app.testing = True
        self.app = my_app.app.test_client()

    def test_home(self):
        result = self.app.get('/')
        # Make your assertions

You can then execute the test cases using the python -m unittest discover command.

More information is available at the Flask Documentation Website.

More Advanced Testing Scenarios

Before you step into creating tests for your application, remember the three basic steps of every test:

  1. Create your inputs
  2. Execute the code, capturing the output
  3. Compare the output with an expected result

It’s not always as easy as creating a static value for the input like a string or a number. Sometimes, your application will require an instance of a class or a context. What do you do then?

The data that you create as an input is known as a fixture. It’s common practice to create fixtures and reuse them.

If you’re running the same test and passing different values each time and expecting the same result, this is known as parameterization.

Handling Expected Failures

Earlier, when you made a list of scenarios to test sum(), a question came up: What happens when you provide it with a bad value, such as a single integer or a string?

In this case, you would expect sum() to throw an error. When it does throw an error, that would cause the test to fail.

There’s a special way to handle expected errors. You can use .assertRaises() as a context-manager, then inside the with block execute the test steps:

import unittest

from my_sum import sum


class TestSum(unittest.TestCase):
    def test_list_int(self):
        """
        Test that it can sum a list of integers
        """
        data = [1, 2, 3]
        result = sum(data)
        self.assertEqual(result, 6)

    def test_list_fraction(self):
        """
        Test that it can sum a list of fractions
        """
        data = [Fraction(1, 4), Fraction(1, 4), Fraction(2, 5)]
        result = sum(data)
        self.assertEqual(result, 1)

    def test_bad_type(self):
        data = "banana"
        with self.assertRaises(TypeError):
            result = sum(data)

if __name__ == '__main__':
    unittest.main()

This test case will now only pass if sum(data) raises a TypeError. You can replace TypeError with any exception type you choose.

Isolating Behaviors in Your Application

Earlier in the tutorial, you learned what a side effect is. Side effects make unit testing harder since, each time a test is run, it might give a different result, or even worse, one test could impact the state of the application and cause another test to fail!

There are some simple techniques you can use to test parts of your application that have many side effects:

  • Refactoring code to follow the Single Responsibility Principle
  • Mocking out any method or function calls to remove side effects
  • Using integration testing instead of unit testing for this piece of the application

Writing Integration Tests

So far, you’ve been learning mainly about unit testing. Unit testing is a great way to build predictable and stable code. But at the end of the day, your application needs to work when it starts!

Integration testing is the testing of multiple components of the application to check that they work together. Integration testing might require acting like a consumer or user of the application by:

  • Calling an HTTP REST API
  • Calling a Python API
  • Calling a web service
  • Running a command line

Each of these types of integration tests can be written in the same way as a unit test, following the Input, Execute, and Assert pattern. The most significant difference is that integration tests are checking more components at once and therefore will have more side effects than a unit test. Also, integration tests will require more fixtures to be in place, like a database, a network socket, or a configuration file.

This is why it’s good practice to separate your unit tests and your integration tests. The creation of fixtures required for an integration like a test database and the test cases themselves often take a lot longer to execute than unit tests, so you may only want to run integration tests before you push to production instead of once on every commit.

A simple way to separate unit and integration tests is simply to put them in different folders:

project/
│
├── my_app/
│   └── __init__.py
│
└── tests/
    |
    ├── unit/
    |   ├── __init__.py
    |   └── test_sum.py
    |
    └── integration/
        ├── __init__.py
        └── test_integration.py

There are many ways to execute only a select group of tests. The specify source directory flag, -s, can be added to unittest discover with the path containing the tests:

$ python -m unittest discover -s tests/integration

unittest will have given you the results of all the tests within the tests/integration directory.

Testing Data-Driven Applications

Many integration tests will require backend data like a database to exist with certain values. For example, you might want to have a test that checks that the application displays correctly with more than 100 customers in the database, or the order page works even if the product names are displayed in Japanese.

These types of integration tests will depend on different test fixtures to make sure they are repeatable and predictable.

A good technique to use is to store the test data in a folder within your integration testing folder called fixtures to indicate that it contains test data. Then, within your tests, you can load the data and run the test.

Here’s an example of that structure if the data consisted of JSON files:

project/
│
├── my_app/
│   └── __init__.py
│
└── tests/
    |
    └── unit/
    |   ├── __init__.py
    |   └── test_sum.py
    |
    └── integration/
        |
        ├── fixtures/
        |   ├── test_basic.json
        |   └── test_complex.json
        |
        ├── __init__.py
        └── test_integration.py

Within your test case, you can use the .setUp() method to load the test data from a fixture file in a known path and execute many tests against that test data. Remember you can have multiple test cases in a single Python file, and the unittest discovery will execute both. You can have one test case for each set of test data:

import unittest


class TestBasic(unittest.TestCase):
    def setUp(self):
        # Load test data
        self.app = App(database='fixtures/test_basic.json')

    def test_customer_count(self):
        self.assertEqual(len(self.app.customers), 100)

    def test_existence_of_customer(self):
        customer = self.app.get_customer(id=10)
        self.assertEqual(customer.name, "Org XYZ")
        self.assertEqual(customer.address, "10 Red Road, Reading")


class TestComplexData(unittest.TestCase):
    def setUp(self):
        # load test data
        self.app = App(database='fixtures/test_complex.json')

    def test_customer_count(self):
        self.assertEqual(len(self.app.customers), 10000)

    def test_existence_of_customer(self):
        customer = self.app.get_customer(id=9999)
        self.assertEqual(customer.name, u"バナナ")
        self.assertEqual(customer.address, "10 Red Road, Akihabara, Tokyo")

if __name__ == '__main__':
    unittest.main()

If your application depends on data from a remote location, like a remote API, you’ll want to ensure your tests are repeatable. Having your tests fail because the API is offline or there is a connectivity issue could slow down development. In these types of situations, it is best practice to store remote fixtures locally so they can be recalled and sent to the application.

The requests library has a complimentary package called responses that gives you ways to create response fixtures and save them in your test folders. Find out more on their GitHub Page.

Testing in Multiple Environments

So far, you’ve been testing against a single version of Python using a virtual environment with a specific set of dependencies. You might want to check that your application works on multiple versions of Python, or multiple versions of a package. Tox is an application that automates testing in multiple environments.

Installing Tox

Tox is available on PyPI as a package to install via pip:

$ pip install tox

Now that you have Tox installed, it needs to be configured.

Configuring Tox for Your Dependencies

Tox is configured via a configuration file in your project directory. The Tox configuration file contains the following:

  • The command to run in order to execute tests
  • Any additional packages required before executing
  • The target Python versions to test against

Instead of having to learn the Tox configuration syntax, you can get a head start by running the quickstart application:

$ tox-quickstart

The Tox configuration tool will ask you those questions and create a file similar to the following in tox.ini:

[tox]
envlist = py27, py36

[testenv]
deps =

commands =
    python -m unittest discover

Before you can run Tox, it requires that you have a setup.py file in your application folder containing the steps to install your package. If you don’t have one, you can follow this guide on how to create a setup.py before you continue.

Alternatively, if your project is not for distribution on PyPI, you can skip this requirement by adding the following line in the tox.ini file under the [tox] heading:

[tox]
envlist = py27, py36
skipsdist=True

If you don’t create a setup.py, and your application has some dependencies from PyPI, you’ll need to specify those on a number of lines under the [testenv] section. For example, Django would require the following:

[testenv]
deps = django

Once you have completed that stage, you’re ready to run the tests.

You can now execute Tox, and it will create two virtual environments: one for Python 2.7 and one for Python 3.6. The Tox directory is called .tox/. Within the .tox/ directory, Tox will execute python -m unittest discover against each virtual environment.

You can run this process by calling Tox at the command line:

$ tox

Tox will output the results of your tests against each environment. The first time it runs, Tox takes a little bit of time to create the virtual environments, but once it has, the second execution will be a lot faster.

Executing Tox

The output of Tox is quite straightforward. It creates an environment for each version, installs your dependencies, and then runs the test commands.

There are some additional command line options that are great to remember.

Run only a single environment, such as Python 3.6:

$ tox -e py36

Recreate the virtual environments, in case your dependencies have changed or site-packages is corrupt:

$ tox -r

Run Tox with less verbose output:

$ tox -q

Running Tox with more verbose output:

$ tox -v

More information on Tox can be found at the Tox Documentation Website.

Automating the Execution of Your Tests

So far, you have been executing the tests manually by running a command. There are some tools for executing tests automatically when you make changes and commit them to a source-control repository like Git. Automated testing tools are often known as CI/CD tools, which stands for “Continuous Integration/Continuous Deployment.” They can run your tests, compile and publish any applications, and even deploy them into production.

Travis CI is one of many available CI (Continuous Integration) services available.

Travis CI works nicely with Python, and now that you’ve created all these tests, you can automate the execution of them in the cloud! Travis CI is free for any open-source projects on GitHub and GitLab and is available for a charge for private projects.

To get started, login to the website and authenticate with your GitHub or GitLab credentials. Then create a file called .travis.yml with the following contents:

language: python
python:
  - "2.7"
  - "3.7"
install:
  - pip install -r requirements.txt
script:
  - python -m unittest discover

This configuration instructs Travis CI to:

  1. Test against Python 2.7 and 3.7 (You can replace those versions with any you choose.)
  2. Install all the packages you list in requirements.txt (You should remove this section if you don’t have any dependencies.)
  3. Run python -m unittest discover to run the tests

Once you have committed and pushed this file, Travis CI will run these commands every time you push to your remote Git repository. You can check out the results on their website.

What’s Next

Now that you’ve learned how to create tests, execute them, include them in your project, and even execute them automatically, there are a few advanced techniques you might find handy as your test library grows.

Introducing Linters Into Your Application

Tox and Travis CI have configuration for a test command. The test command you have been using throughout this tutorial is python -m unittest discover.

You can provide one or many commands in all of these tools, and this option is there to enable you to add more tools that improve the quality of your application.

One such type of application is called a linter. A linter will look at your code and comment on it. It could give you tips about mistakes you’ve made, correct trailing spaces, and even predict bugs you may have introduced.

Passive Linting With flake8

A popular linter that comments on the style of your code in relation to the PEP 8 specification is flake8.

You can install flake8 using pip:

$ pip install flake8

You can then run flake8 over a single file, a folder, or a pattern:

$ flake8 test.py
test.py:6:1: E302 expected 2 blank lines, found 1
test.py:23:1: E305 expected 2 blank lines after class or function definition, found 1
test.py:24:20: W292 no newline at end of file

You will see a list of errors and warnings for your code that flake8 has found.

flake8 is configurable on the command line or inside a configuration file in your project. If you wanted to ignore certain rules, like E305 shown above, you can set them in the configuration. flake8 will inspect a .flake8 file in the project folder or a setup.cfg file. If you decided to use Tox, you can put the flake8 configuration section inside tox.ini.

This example ignores the .git and __pycache__ directories as well as the E305 rule. Also, it sets the max line length to 90 instead of 80 characters. You will likely find that the default constraint of 79 characters for line-width is very limiting for tests, as they contain long method names, string literals with test values, and other pieces of data that can be longer. It is common to set the line length for tests to up to 120 characters:

[flake8]
ignore = E305
exclude = .git,__pycache__
max-line-length = 90

Alternatively, you can provide these options on the command line:

$ flake8 --ignore E305 --exclude .git,__pycache__ --max-line-length=90

A full list of configuration options is available on the Documentation Website.

You can now add flake8 to your CI configuration. For Travis CI, this would look as follows:

matrix:
  include:
    - python: "2.7"
      script: "flake8"

Travis will read the configuration in .flake8 and fail the build if any linting errors occur. Be sure to add the flake8 dependency to your requirements.txt file.

Aggressive Linting With a Code Formatter

flake8 is a passive linter: it recommends changes, but you have to go and change the code. A more aggressive approach is a code formatter. Code formatters will change your code automatically to meet a collection of style and layout practices.

black is a very unforgiving formatter. It doesn’t have any configuration options, and it has a very specific style. This makes it great as a drop-in tool to put in your test pipeline.

Note: black requires Python 3.6+.

You can install black via pip:

$ pip install black

Then to run black at the command line, provide the file or directory you want to format:

$ black test.py

Keeping Your Test Code Clean

When writing tests, you may find that you end up copying and pasting code a lot more than you would in regular applications. Tests can be very repetitive at times, but that is by no means a reason to leave your code sloppy and hard to maintain.

Over time, you will develop a lot of technical debt in your test code, and if you have significant changes to your application that require changes to your tests, it can be a more cumbersome task than necessary because of the way you structured them.

Try to follow the DRY principle when writing tests: Don’t Repeat Yourself.

Test fixtures and functions are a great way to produce test code that is easier to maintain. Also, readability counts. Consider deploying a linting tool like flake8 over your test code:

$ flake8 --max-line-length=120 tests/

Testing for Performance Degradation Between Changes

There are many ways to benchmark code in Python. The standard library provides the timeit module, which can time functions a number of times and give you the distribution. This example will execute test() 100 times and print() the output:

def test():
    # ... your code

if __name__ == '__main__':
    import timeit
    print(timeit.timeit("test()", setup="from __main__ import test", number=100))

Another option, if you decided to use pytest as a test runner, is the pytest-benchmark plugin. This provides a pytest fixture called benchmark. You can pass benchmark() any callable, and it will log the timing of the callable to the results of pytest.

You can install pytest-benchmark from PyPI using pip:

$ pip install pytest-benchmark

Then, you can add a test that uses the fixture and passes the callable to be executed:

def test_my_function(benchmark):
    result = benchmark(test)

Execution of pytest will now give you benchmark results:

More information is available at the Documentation Website.

Testing for Security Flaws in Your Application

Another test you will want to run on your application is checking for common security mistakes or vulnerabilities.

You can install bandit from PyPI using pip:

$ pip install bandit

You can then pass the name of your application module with the -r flag, and it will give you a summary:

$ bandit -r my_sum
[main]  INFO    profile include tests: None
[main]  INFO    profile exclude tests: None
[main]  INFO    cli include tests: None
[main]  INFO    cli exclude tests: None
[main]  INFO    running on Python 3.5.2
Run started:2018-10-08 00:35:02.669550

Test results:
        No issues identified.

Code scanned:
        Total lines of code: 5
        Total lines skipped (#nosec): 0

Run metrics:
        Total issues (by severity):
                Undefined: 0.0
                Low: 0.0
                Medium: 0.0
                High: 0.0
        Total issues (by confidence):
                Undefined: 0.0
                Low: 0.0
                Medium: 0.0
                High: 0.0
Files skipped (0):

As with flake8, the rules that bandit flags are configurable, and if there are any you wish to ignore, you can add the following section to your setup.cfg file with the options:

[bandit]
exclude: /test
tests: B101,B102,B301	

More details are available at the GitHub Website.

Conclusion

Python has made testing accessible by building in the commands and libraries you need to validate that your applications work as designed. Getting started with testing in Python needn’t be complicated: you can use unittest and write small, maintainable methods to validate your code.

As you learn more about testing and your application grows, you can consider switching to one of the other test frameworks, like pytest, and start to leverage more advanced features.

Thank you for reading. I hope you have a bug-free future with Python!

Top 5 Python Frameworks For Test Automation In 2019

Top 5 Python Frameworks For Test Automation In 2019

In this article, we'll list top 5 Python Frameworks for Test Automation in 2019.

After being voted as the best programming language in the year 2018, Python still continues rising up the charts and currently ranks as the 3rd best programming language just after Java and C, as per the index published by Tiobe. With the increasing use of this language, the popularity of test automation frameworks based on Python is increasing as well. Obviously, developers and testers will get a little bit confused when it comes to choosing the best framework for their project. While choosing one, you should judge a lot of things, the script quality of the framework, test case simplicity and the technique to run the modules and find out their weaknesses. This is my attempt to help you compare the top 5 Python frameworks for test automation in 2019, and their advantages over the other as well as disadvantages. So you could choose the ideal Python framework for test automation according to your needs.

Robot Framework

Used mostly for development that is acceptance test-driven as well as for acceptance testing, Robot Framework is one of the top Python test frameworks. Although it is developed using Python, it can also run on IronPython, which is .net-based and on Java-based Jython. Robot as a Python framework is compatible across all platforms – Windows, MacOS or Linux.

What Are The Prerequisites?

Yes, there are

  • First of all, you will be able to use Robot Framework (RF), only when you have Python 2.7.14 or any version above it installed. Although nowadays, Python 3.6.4 is also used, code snippets provided in the official blog of RF will make sure that appropriate notes are added consisting of all the changes required.
  • You will also need to install ‘pip’ or python package manager.
  • Finally, a development framework should be must to download. A popular framework among developers is the PyCharm Community Edition. However, since code snippets are not IDE-dependent, you can use any IDE which you have worked on earlier.

What Are The Advantages & Disadvantages Of Robot?

Let’s take a look at what are the advantages and disadvantages of Robot as a test automation framework over other Python frameworks.

Pros

  • First of all, you will be able to use Robot Framework (RF), only when you have Python 2.7.14 or any version above it installed. Although nowadays, Python 3.6.4 is also used, code snippets provided in the official blog of RF will make sure that appropriate notes are added consisting of all the changes required.
  • You will also need to install ‘pip’ or python package manager.
  • Finally, a development framework should be must to download. A popular framework among developers is the PyCharm Community Edition. However, since code snippets are not IDE-dependent, you can use any IDE which you have worked on earlier.

Cons

  • First of all, you will be able to use Robot Framework (RF), only when you have Python 2.7.14 or any version above it installed. Although nowadays, Python 3.6.4 is also used, code snippets provided in the official blog of RF will make sure that appropriate notes are added consisting of all the changes required.
  • You will also need to install ‘pip’ or python package manager.
  • Finally, a development framework should be must to download. A popular framework among developers is the PyCharm Community Edition. However, since code snippets are not IDE-dependent, you can use any IDE which you have worked on earlier.

Is Robot The Top Python Test Framework For You?

If you are a beginner in the automation domain and have less experience in development, using Robot as a top Python test framework is easier to use than pytest or pyunit, since it has rich in built libraries and involves using an easier test-oriented DSL. However, if you want to develop a complex automation framework, it is better to switch to pytest or any other framework involving Python code.

If you are new to Robot framework, here is a document that would help you run your first automation script using Robot framework with Selenium.

Pytest

Used for all kinds of software testing, pytest is another top Python test framework for test automation. Being open source and easy to learn, the tool can be used by QA teams, development teams as well as individual practice groups and in open source projects. Because of its useful features like ‘assert rewriting’, most projects on the internet, including big shots like Dropbox and Mozilla, have switched from unittest(Pyunit) to pytest. Let’s take a deep dive and find out what’s so special about this Python framework.

What Are The Prerequisites?

Apart from working knowledge in Python, pytest does not need anything complex. All you need is a working desktop that has a command line interface, python package manager and an IDE for development.

What Are The Advantages & Disadvantages Of pytest?

Pros

  • First of all, you will be able to use Robot Framework (RF), only when you have Python 2.7.14 or any version above it installed. Although nowadays, Python 3.6.4 is also used, code snippets provided in the official blog of RF will make sure that appropriate notes are added consisting of all the changes required.
  • You will also need to install ‘pip’ or python package manager.
  • Finally, a development framework should be must to download. A popular framework among developers is the PyCharm Community Edition. However, since code snippets are not IDE-dependent, you can use any IDE which you have worked on earlier.

Cons

The fact that special routines are used by pytest means that you have to compromise with compatibility. You will be able to conveniently write test cases but you won’t be able to use those test cases with any other testing framework.

Is Pytest The Top Python Test Framework For You?

Well, you have to start by learning a full-fledged language but once you get the hang of it, you will get all the features like static code analysis, support for multiple IDE and most importantly, writing effective test cases. For writing functional test cases and developing a complex framework, it is better than unittest but its advantage is somewhat similar to Robot Framework if your aim is to develop a simple framework.

If you are considering Pytest as a top Python test framework for you then here is a guide to test automation using pytest and Selenium WebDriver.

UnitTest, Also Known As PyUnit

Unittest or PyUnit is the standard test automation framework for unit testing that comes with Python. It’s highly inspired by JUnit. The assertion methods and all the cleanup and setup routines are provided by the base class TestCase. The name of each and every method in the subclass of TestCase starts with “test”. This allows them to run as test cases. You can use the load methods and the TestSuite class to the group and load the tests. Together, you can use them to build customized test runners. Just like Selenium testing with JUnit, unittest also has the ability to use unittest-sml-reporting and generate XML reports.

What Are The Prerequisites?

There are no such prerequisites since unittest comes by default with Python. To use it, you will need standard knowledge of the python framework and also if you want to install additional modules, you will need pip installed along with an IDE for development.

What Are The Advantages & Disadvantages Of PyUnit?

Pros

Being part of the standard library of Python, there are several advantages of using Unittest.

  • First of all, you will be able to use Robot Framework (RF), only when you have Python 2.7.14 or any version above it installed. Although nowadays, Python 3.6.4 is also used, code snippets provided in the official blog of RF will make sure that appropriate notes are added consisting of all the changes required.
  • You will also need to install ‘pip’ or python package manager.
  • Finally, a development framework should be must to download. A popular framework among developers is the PyCharm Community Edition. However, since code snippets are not IDE-dependent, you can use any IDE which you have worked on earlier.

An Introduction to Unit Testing in Python

An Introduction to Unit Testing in Python

Unit tests can pass or fail, and that makes them a great technique to check your code. In this tutorial Goran Aviani will demonstrate how to write unit tests in python

Originally published by Goran Aviani at freecodecamp.org

You just finished writing a piece of code and you are wondering what to do. Will you submit a pull request and have your teammates review the code? Or will you manually test the code?

You should do both of these things, but with an additional step: you need to unit test your code to make sure that the code works as intended.

Getting started

The best way you can understand testing is if you do it hands-on. For that purpose, in a file named name_function.py, I will write a simple function that takes a first and last name, and returns a full name:

#Generate a formatted full name
def formatted_name(first_name, last_name):
   full_name = first_name + ' ' + last_name
   return full_name.title()

The function formatted_name() takes the first and the last name and combines them with a space between to form a full name. It then capitalizes the first letter of every word. To check that this code works, you need to write some code that uses this function. In names.py I will write some simple code that lets users enter their first and last names:

from name_function import formatted_name

print("Please enter the first and last names or enter x to E[x]it.")

while True:
first_name = input("Please enter the first name: ")
if first_name == "x":
print("Good bye.")
break

last_name = input("Please enter the last name: ")
if last_name == "x":
print("Good bye.")
break

result = formatted_name(first_name, last_name)
print("Formatted name is: " + result + ".")

This code imports formatted_name() from name_function.py and on running, allows the user to enter a series of first and last names and shows the formatted full names.

Unit test and Test cases

There is a module in Python’s standard library called unittest which contains tools for testing your code. Unit testing checks if all specific parts of your function’s behavior are correct, which will make integrating them together with other parts much easier.

Test case is a collection of unit tests which together proves that a function works as intended, inside a full range of situations in which that function may find itself and that it’s expected to handle. Test case should consider all possible kinds of input a function could receive from users, and therefore should include tests to represent each of these situations.

Passing a test

Here’s a typical scenario for writing tests:

First you need to create a test file. Then import the unittest module, define the testing class that inherits from unittest.TestCase, and lastly, write a series of methods to test all the cases of your function’s behavior.

There’s a line by line explanation below the following code:

import unittest
from name_function import formatted_name

class NamesTestCase(unittest.TestCase):

def test_first_last_name(self):
result = formatted_name("pete", "seeger")
self.assertEqual(result, "Pete Seeger")

First, you need to import a unittest and the function you want to test, formatted_name() . Then you create a class, for example NamesTestCase, that will contain tests for your formatted_name() function. This class inherits from the class unittest.TestCase.

NamesTestCase contains a single method that tests one part of formatted_name() . You can call this method test_first_last_name().

Remember that every method that starts with “test_” will be run automatically when you run test_name_function.py.

Within test_first_last_name() test method, you call the function you want to test and store a return value. In this example we are going to call formatted_name() with the arguments “pete” and “seeger” , and store the result in the resulting variable.

In the last line we will use the assert method. The assert method verifies that a result you received matches the result you expected to receive. And in this case we know that formatted_name() function will return full name with capitalized first letters, so we expect the result “Pete Seeger”. To check this, the unittest’s assertEqual() method is being used.

self.assertEqual(result, “Pete Seeger”)

This line basically means: Compare the value in the resulting variable with “Pete Seeger” and if they are equal it’s OK, but if they are not let me know.

On running test_name_function.py you are expected to get a OK meaning that the test has passed.

Ran 1 test in 0.001s OK
Failing a test

To show you what a failing test looks like I’m going to modify a formatted_name() function by including a new middle name argument.

So I’m going to rewrite the function to look like this:

#Generate a formatted full name including a middle name
def formatted_name(first_name, last_name, middle_name):
full_name = first_name + ' ' + middle_name + ' ' + last_name
return full_name.title()

This version of formatted_name() will work for people with middle names, but when you test it you will see that the function is broken for people who don’t have a middle name.

So when you run the test_name_function.py you will get the output that looks something like this:

Error
Traceback (most recent call last):
 
File “test_name_function.py”, line 7, in test_first_last_name
    result = formatted_name(“pete”, “seeger”)
 
TypeError: formatted_name() missing 1 required positional argument: ‘middle_name’
 
Ran 1 test in 0.002s
 
FAILED (errors=1)

In the output you will see information that will tell you all you need to know where the test fails:

  • First item in the output is the Error telling you that at least one test in test case resulted in an error.
  • Next you’ll see the file and method in which the error occurred.
  • After that you will see the line in which the error occurred.
  • And what kind of error it is, in this case we are missing 1 argument “middle_name”.
  • You will also see the number of run tests, the time needed for the tests to complete, and a textual message that represents the status of the tests with number of errors that occurred.
What to do when the test has failed
A passing test means the function is behaving according to what’s expected from it. However, a failing test means there’s more fun ahead of you.

I’ve seen couple of programmers that prefer to change the test instead of improving the code — but don’t to that. Spend a little more time to fix the issue, as it will help you to better understand the code and save time in the long run.

In this example, our function formatted_name() first required two parameters, and now as it is rewritten it requires one extra: a middle name. Adding a middle name to our function broke the desired behavior of it. Since the idea is not to make changes to the tests, the best solution is to make middle name optional.

After we do this the idea is to make the tests pass when the first and last name are used, for example “Pete Seeger”, as well as when first, last and middle names are used, for example “Raymond Red Reddington”. So let’s modify the code of formatted_name() once again:

#Generate a formatted full name including a middle name
def formatted_name(first_name, last_name, middle_name=''):
   if len(middle_name) > 0:
       full_name = first_name + ' ' + middle_name + ' ' + last_name
   else:
       full_name = first_name + ' ' + last_name
   return full_name.title()

Now the function should work for names with and without the middle name.

And to make sure it still works with “Pete Seeger” run the test again:

Ran 1 test in 0.001s

OK

And this is what I intended to show you: It’s always better to make changes to your code to fit your tests than other way around. Now the time has come to add a new test for names that do have a middle name.
Adding new tests

Write a new method to the NamesTestCase class that will test for middle names:

import unittest
from name_function import formatted_name
 
class NamesTestCase(unittest.TestCase):
 
    def test_first_last_name(self):
        result = formatted_name("pete", "seeger")
        self.assertEqual(result, "Pete Seeger")
 
    def test_first_last_middle_name(self):
        result = formatted_name("raymond", "reddington", "red")
        self.assertEqual(result, "Raymond Red Reddington")

After you run the test, both tests should pass:

Ran 2 tests in 0.001s

OK

Bra gjort!
Well done!

You have written your tests to check if the function works using names with or without a middle name. 

Originally published by Goran Aviani at freecodecamp.org

 ===================================================================

Thanks for reading :heart: If you liked this post, share it with all of your programming buddies! Follow me on Facebook | Twitter

Learn More

☞ Complete Python Bootcamp: Go from zero to hero in Python 3

☞ Python for Time Series Data Analysis

☞ The complete beginner’s guide to JSON

☞ The Complete Guide to JSON Web Tokens

☞ Python Programming For Beginners From Scratch

☞ Python Network Programming | Network Apps & Hacking Tools

☞ Intro To SQLite Databases for Python Programming

☞ Ethical Hacking With Python, JavaScript and Kali Linux

☞ Beginner’s guide on Python: Learn python from scratch! (New)

☞ Python for Beginners: Complete Python Programming


What's Python IDLE? How to use Python IDLE to interact with Python?

What's Python IDLE? How to use Python IDLE to interact with Python?

In this tutorial, you’ll learn all the basics of using **IDLE** to write Python programs. You'll know what Python IDLE is and how you can use it to interact with Python directly. You’ve also learned how to work with Python files and customize Python IDLE to your liking.

In this tutorial, you'll learn how to use the development environment included with your Python installation. Python IDLE is a small program that packs a big punch! You'll learn how to use Python IDLE to interact with Python directly, work with Python files, and improve your development workflow.

If you’ve recently downloaded Python onto your computer, then you may have noticed a new program on your machine called IDLE. You might be wondering, “What is this program doing on my computer? I didn’t download that!” While you may not have downloaded this program on your own, IDLE comes bundled with every Python installation. It’s there to help you get started with the language right out of the box. In this tutorial, you’ll learn how to work in Python IDLE and a few cool tricks you can use on your Python journey!

In this tutorial, you’ll learn:

  • What Python IDLE is
  • How to interact with Python directly using IDLE
  • How to edit, execute, and debug Python files with IDLE
  • How to customize Python IDLE to your liking

Table of Contents

What Is Python IDLE?

Every Python installation comes with an Integrated Development and Learning Environment, which you’ll see shortened to IDLE or even IDE. These are a class of applications that help you write code more efficiently. While there are many IDEs for you to choose from, Python IDLE is very bare-bones, which makes it the perfect tool for a beginning programmer.

Python IDLE comes included in Python installations on Windows and Mac. If you’re a Linux user, then you should be able to find and download Python IDLE using your package manager. Once you’ve installed it, you can then use Python IDLE as an interactive interpreter or as a file editor.

An Interactive Interpreter

The best place to experiment with Python code is in the interactive interpreter, otherwise known as a shell. The shell is a basic Read-Eval-Print Loop (REPL). It reads a Python statement, evaluates the result of that statement, and then prints the result on the screen. Then, it loops back to read the next statement.

The Python shell is an excellent place to experiment with small code snippets. You can access it through the terminal or command line app on your machine. You can simplify your workflow with Python IDLE, which will immediately start a Python shell when you open it.

A File Editor

Every programmer needs to be able to edit and save text files. Python programs are files with the .py extension that contain lines of Python code. Python IDLE gives you the ability to create and edit these files with ease.

Python IDLE also provides several useful features that you’ll see in professional IDEs, like basic syntax highlighting, code completion, and auto-indentation. Professional IDEs are more robust pieces of software and they have a steep learning curve. If you’re just beginning your Python programming journey, then Python IDLE is a great alternative!

How to Use the Python IDLE Shell

The shell is the default mode of operation for Python IDLE. When you click on the icon to open the program, the shell is the first thing that you see:

This is a blank Python interpreter window. You can use it to start interacting with Python immediately. You can test it out with a short line of code:

Here, you used print() to output the string "Hello, from IDLE!" to your screen. This is the most basic way to interact with Python IDLE. You type in commands one at a time and Python responds with the result of each command.

Next, take a look at the menu bar. You’ll see a few options for using the shell:

You can restart the shell from this menu. If you select that option, then you’ll clear the state of the shell. It will act as though you’ve started a fresh instance of Python IDLE. The shell will forget about everything from its previous state:

In the image above, you first declare a variable, x = 5. When you call print(x), the shell shows the correct output, which is the number 5. However, when you restart the shell and try to call print(x) again, you can see that the shell prints a traceback. This is an error message that says the variable x is not defined. The shell has forgotten about everything that came before it was restarted.

You can also interrupt the execution of the shell from this menu. This will stop any program or statement that’s running in the shell at the time of interruption. Take a look at what happens when you send a keyboard interrupt to the shell:

A KeyboardInterrupt error message is displayed in red text at the bottom of your window. The program received the interrupt and has stopped executing.

How to Work With Python Files

Python IDLE offers a full-fledged file editor, which gives you the ability to write and execute Python programs from within this program. The built-in file editor also includes several features, like code completion and automatic indentation, that will speed up your coding workflow. First, let’s take a look at how to write and execute programs in Python IDLE.

Opening a File

To start a new Python file, select File → New File from the menu bar. This will open a blank file in the editor, like this:

From this window, you can write a brand new Python file. You can also open an existing Python file by selecting File → Open… in the menu bar. This will bring up your operating system’s file browser. Then, you can find the Python file you want to open.

If you’re interested in reading the source code for a Python module, then you can select File → Path Browser. This will let you view the modules that Python IDLE can see. When you double click on one, the file editor will open up and you’ll be able to read it.

The content of this window will be the same as the paths that are returned when you call sys.path. If you know the name of a specific module you want to view, then you can select File → Module Browser and type in the name of the module in the box that appears.

Editing a File

Once you’ve opened a file in Python IDLE, you can then make changes to it. When you’re ready to edit a file, you’ll see something like this:

The contents of your file are displayed in the open window. The bar along the top of the window contains three pieces of important information:

  1. The name of the file that you’re editing
  2. The full path to the folder where you can find this file on your computer
  3. The version of Python that IDLE is using

In the image above, you’re editing the file myFile.py, which is located in the Documents folder. The Python version is 3.7.1, which you can see in parentheses.

There are also two numbers in the bottom right corner of the window:

  1. Ln: shows the line number that your cursor is on.
  2. Col: shows the column number that your cursor is on.

It’s useful to see these numbers so that you can find errors more quickly. They also help you make sure that you’re staying within a certain line width.

There are a few visual cues in this window that will help you remember to save your work. If you look closely, then you’ll see that Python IDLE uses asterisks to let you know that your file has unsaved changes:

The file name shown in the top of the IDLE window is surrounded by asterisks. This means that there are unsaved changes in your editor. You can save these changes with your system’s standard keyboard shortcut, or you can select File → Save from the menu bar. Make sure that you save your file with the .py extension so that syntax highlighting will be enabled.

Executing a File

When you want to execute a file that you’ve created in IDLE, you should first make sure that it’s saved. Remember, you can see if your file is properly saved by looking for asterisks around the filename at the top of the file editor window. Don’t worry if you forget, though! Python IDLE will remind you to save whenever you attempt to execute an unsaved file.

To execute a file in IDLE, simply press the F5 key on your keyboard. You can also select Run → Run Module from the menu bar. Either option will restart the Python interpreter and then run the code that you’ve written with a fresh interpreter. The process is the same as when you run python3 -i [filename] in your terminal.

When your code is done executing, the interpreter will know everything about your code, including any global variables, functions, and classes. This makes Python IDLE a great place to inspect your data if something goes wrong. If you ever need to interrupt the execution of your program, then you can press Ctrl+C in the interpreter that’s running your code.

How to Improve Your Workflow

Now that you’ve seen how to write, edit, and execute files in Python IDLE, it’s time to speed up your workflow! The Python IDLE editor offers a few features that you’ll see in most professional IDEs to help you code faster. These features include automatic indentation, code completion and call tips, and code context.

Automatic Indentation

IDLE will automatically indent your code when it needs to start a new block. This usually happens after you type a colon (:). When you hit the enter key after the colon, your cursor will automatically move over a certain number of spaces and begin a new code block.

You can configure how many spaces the cursor will move in the settings, but the default is the standard four spaces. The developers of Python agreed on a standard style for well-written Python code, and this includes rules on indentation, whitespace, and more. This standard style was formalized and is now known as PEP 8. To learn more about it, check out How to Write Beautiful Python Code With PEP 8.

Code Completion and Call Tips

When you’re writing code for a large project or a complicated problem, you can spend a lot of time just typing out all of the code you need. Code completion helps you save typing time by trying to finish your code for you. Python IDLE has basic code completion functionality. It can only autocomplete the names of functions and classes. To use autocompletion in the editor, just press the tab key after a sequence of text.

Python IDLE will also provide call tips. A call tip is like a hint for a certain part of your code to help you remember what that element needs. After you type the left parenthesis to begin a function call, a call tip will appear if you don’t type anything for a few seconds. For example, if you can’t quite remember how to append to a list, then you can pause after the opening parenthesis to bring up the call tip:

The call tip will display as a popup note, reminding you how to append to a list. Call tips like these provide useful information as you’re writing code.

Code Context

The code context functionality is a neat feature of the Python IDLE file editor. It will show you the scope of a function, class, loop, or other construct. This is particularly useful when you’re scrolling through a lengthy file and need to keep track of where you are while reviewing code in the editor.

To turn it on, select Options → Code Context in the menu bar. You’ll see a gray bar appear at the top of the editor window:

As you scroll down through your code, the context that contains each line of code will stay inside of this gray bar. This means that the print() functions you see in the image above are a part of a main function. When you reach a line that’s outside the scope of this function, the bar will disappear.

How to Debug in IDLE

A bug is an unexpected problem in your program. They can appear in many forms, and some are more difficult to fix than others. Some bugs are tricky enough that you won’t be able to catch them by just reading through your program. Luckily, Python IDLE provides some basic tools that will help you debug your programs with ease!

Interpreter DEBUG Mode

If you want to run your code with the built-in debugger, then you’ll need to turn this feature on. To do so, select Debug → Debugger from the Python IDLE menu bar. In the interpreter, you should see [DEBUG ON] appear just before the prompt (>>>), which means the interpreter is ready and waiting.

When you execute your Python file, the debugger window will appear:

In this window, you can inspect the values of your local and global variables as your code executes. This gives you insight into how your data is being manipulated as your code runs.

You can also click the following buttons to move through your code:

  • Go: Press this to advance execution to the next breakpoint. You’ll learn about these in the next section.
  • Step: Press this to execute the current line and go to the next one.
  • Over: If the current line of code contains a function call, then press this to step over that function. In other words, execute that function and go to the next line, but don’t pause while executing the function (unless there is a breakpoint).
  • Out: If the current line of code is in a function, then press this to step out of this function. In other words, continue the execution of this function until you return from it.

Be careful, because there is no reverse button! You can only step forward in time through your program’s execution.

You’ll also see four checkboxes in the debug window:

  1. Globals: your program’s global information
  2. Locals: your program’s local information during execution
  3. Stack: the functions that run during execution
  4. Source: your file in the IDLE editor

When you select one of these, you’ll see the relevant information in your debug window.

Breakpoints

A breakpoint is a line of code that you’ve identified as a place where the interpreter should pause while running your code. They will only work when DEBUG mode is turned on, so make sure that you’ve done that first.

To set a breakpoint, right-click on the line of code that you wish to pause. This will highlight the line of code in yellow as a visual indication of a set breakpoint. You can set as many breakpoints in your code as you like. To undo a breakpoint, right-click the same line again and select Clear Breakpoint.

Once you’ve set your breakpoints and turned on DEBUG mode, you can run your code as you would normally. The debugger window will pop up, and you can start stepping through your code manually.

Errors and Exceptions

When you see an error reported to you in the interpreter, Python IDLE lets you jump right to the offending file or line from the menu bar. All you have to do is highlight the reported line number or file name with your cursor and select Debug → Go to file/line from the menu bar. This is will open up the offending file and take you to the line that contains the error. This feature works regardless of whether or not DEBUG mode is turned on.

Python IDLE also provides a tool called a stack viewer. You can access it under the Debug option in the menu bar. This tool will show you the traceback of an error as it appears on the stack of the last error or exception that Python IDLE encountered while running your code. When an unexpected or interesting error occurs, you might find it helpful to take a look at the stack. Otherwise, this feature can be difficult to parse and likely won’t be useful to you unless you’re writing very complicated code.

How to Customize Python IDLE

There are many ways that you can give Python IDLE a visual style that suits you. The default look and feel is based on the colors in the Python logo. If you don’t like how anything looks, then you can almost always change it.

To access the customization window, select Options → Configure IDLE from the menu bar. To preview the result of a change you want to make, press Apply. When you’re done customizing Python IDLE, press OK to save all of your changes. If you don’t want to save your changes, then simply press Cancel.

There are 5 areas of Python IDLE that you can customize:

  1. Fonts/Tabs
  2. Highlights
  3. Keys
  4. General
  5. Extensions

Let’s take a look at each of them now.

Fonts/Tabs

The first tab allows you to change things like font color, font size, and font style. You can change the font to almost any style you like, depending on what’s available for your operating system. The font settings window looks like this:

You can use the scrolling window to select which font you prefer. (I recommend you select a fixed-width font like Courier New.) Pick a font size that’s large enough for you to see well. You can also click the checkbox next to Bold to toggle whether or not all text appears in bold.

This window will also let you change how many spaces are used for each indentation level. By default, this will be set to the PEP 8 standard of four spaces. You can change this to make the width of your code more or less spread out to your liking.

Highlights

The second customization tab will let you change highlights. Syntax highlighting is an important feature of any IDE that highlights the syntax of the language that you’re working in. This helps you visually distinguish between the different Python constructs and the data used in your code.

Python IDLE allows you to fully customize the appearance of your Python code. It comes pre-installed with three different highlight themes:

  1. IDLE Day
  2. IDLE Night
  3. IDLE New

You can select from these pre-installed themes or create your own custom theme right in this window:

Unfortunately, IDLE does not allow you to install custom themes from a file. You have to create customs theme from this window. To do so, you can simply start changing the colors for different items. Select an item, and then press Choose color for. You’ll be brought to a color picker, where you can select the exact color that you want to use.

You’ll then be prompted to save this theme as a new custom theme, and you can enter a name of your choosing. You can then continue changing the colors of different items if you’d like. Remember to press Apply to see your changes in action!

Keys

The third customization tab lets you map different key presses to actions, also known as keyboard shortcuts. These are a vital component of your productivity whenever you use an IDE. You can either come up with your own keyboard shortcuts, or you can use the ones that come with IDLE. The pre-installed shortcuts are a good place to start:

The keyboard shortcuts are listed in alphabetical order by action. They’re listed in the format Action - Shortcut, where Action is what will happen when you press the key combination in Shortcut. If you want to use a built-in key set, then select a mapping that matches your operating system. Pay close attention to the different keys and make sure your keyboard has them!

Creating Your Own Shortcuts

The customization of the keyboard shortcuts is very similar to the customization of syntax highlighting colors. Unfortunately, IDLE does not allow you to install custom keyboard shortcuts from a file. You must create a custom set of shortcuts from the Keys tab.

Select one pair from the list and press Get New Keys for Selection. A new window will pop up:

Here, you can use the checkboxes and scrolling menu to select the combination of keys that you want to use for this shortcut. You can select Advanced Key Binding Entry >> to manually type in a command. Note that this cannot pick up the keys you press. You have to literally type in the command as you see it displayed to you in the list of shortcuts.

General

The fourth tab of the customization window is a place for small, general changes. The general settings tab looks like this:

Here, you can customize things like the window size and whether the shell or the file editor opens first when you start Python IDLE. Most of the things in this window are not that exciting to change, so you probably won’t need to fiddle with them much.

Extensions

The fifth tab of the customization window lets you add extensions to Python IDLE. Extensions allow you to add new, awesome features to the editor and the interpreter window. You can download them from the internet and install them to right into Python IDLE.

To view what extensions are installed, select Options → Configure IDLE -> Extensions. There are many extensions available on the internet for you to read more about. Find the ones you like and add them to Python IDLE!

Conclusion

In this tutorial, you’ve learned all the basics of using IDLE to write Python programs. You know what Python IDLE is and how you can use it to interact with Python directly. You’ve also learned how to work with Python files and customize Python IDLE to your liking.

You’ve learned how to:

  • Work with the Python IDLE shell
  • Use Python IDLE as a file editor
  • Improve your workflow with features to help you code faster
  • Debug your code and view errors and exceptions
  • Customize Python IDLE to your liking

Now you’re armed with a new tool that will let you productively write Pythonic code and save you countless hours down the road. Happy programming!