Article Thumbnail

How To Debug Your Python Code

A gentle introduction to debugging in Python

Florian Dahlitz
11 min
May 11, 2020

Introduction

Throughout your day to day coding sessions, you come across bugs in your or other's software. Although most of the people use print-statements in their language of choice for debugging purposes, they cannot replace a debugger.

As I am also a print-statement guy, but know the benefits of a good debugger, I wanted to learn more about debugging in Python. With this article, I take you with me on this journey - enjoy!

Note: You can find the code examples used throughout the article on GitHub.

What Is a Debugger?

This article gives you a gentle introduction to debugging in Python. But before we can start investigating what Python already provides us to debug code, we first need to understand what a debugger is. Wikipedia says:

"A debugger or debugging tool is a computer program used to test and debug other programs [...]. The main use of a debugger is to run the target program under controlled conditions that permit the programmer to track its operations in progress and monitor changes in computer resources [...] that may indicate malfunctioning code." [1]

In other words, a debugger is a piece of software you use to manually test your code and find bugs.

What Does Python Provide?

Python comes with its own debugger framework called bdb [2]. The idea is to implement your own debugger based on the framework provided. The bdb module handles basic debugger functions, like setting breakpoints or managing execution via the debugger. It defines an exception BdbQuit, which is raised by the Bdb class for quitting the debugger, as well as two classes Breakpoint and Bdb.

An instance of the Breakpoint class represents a single breakpoint in your code. You can think of a breakpoint like a pause in the execution. If the debugger hits a breakpoint, the execution stops and you are able to inspect the environment you are currently in. This includes variables that are currently defined as well as actual values, constants, and more. If you expect to see a certain state at this point but find another one, you may have found a bug.

The Bdb class acts as a generic Python debugger base class and takes care of the details of the trace facility. However, derived classes should implement user interaction as Bdb is not providing support for it out of the box.

Besides the debugger framework, Python provides an actual debugger called pdb, which we will have a closer look at.

Python's Pdb Module

I do not want to simply write down the specifications written in the Python documentation. In my humble opinion, this is not going to help you and me to understand the topic. Instead, I want to show you a practical example and how to debug the code using pdb. While we wade through the examples, you will get to know the specifications, too.

Without further introduction, let's have a look at some examples.

Breakpoints and Pdb Commands

The first example is a simple script called modify.py.

# modify.py

import pdb

z = 2_000


def modify(x: int, y: int):
    global z
    y, z = z, y
    z += x + y


height, width = 720, 1080
pdb.set_trace()
modify(height, width)

At the beginning of the modify.py script, we import the pdb module and define a global variable z.

Next, a function called modify() is implemented. It accepts exactly two arguments x and y. Both are integers. We need to tell Python that z is a global variable and can be found outside the scope of the function using Python's global keyword. Otherwise, we would get an error on the next line, when we swap the values of y and z, telling us that z is referenced before it got anything assigned to. Subsequently, we add the sum of x and y to z.

Besides the function and the first global variable, we define two variables height and width and assign standard display resolutions to them. Now, we are at an interesting point: We set a breakpoint by using the set_trace() function of the pdb module. Afterwards, modify() is called with the variables height and width as arguments.

If we execute the script at hand without the breakpoint inserted by set_trace(), the script would just terminate normally without printing anything. However, executing the script as it is results in:

$ python modify.py
> /home/florian/workspace/python/debugging-in-python/modify.py(16)<module>()
-> modify(height, width)
(Pdb)

So what is going on here? The first line, which starts with a > symbol indicates the location where the breakpoint that was hit is located. The path to the file is specified as well as the line where the breakpoint is located and the scope. In this case, the scope is module. If the breakpoint is inside a function, the function name would be displayed.

Note: The line number is 16, although pdb.set_trace() is called in line 15. Technically, the breakpoint is exactly at the beginning of line 16 so before line 16 is executed. Keep this in mind.

The next line, which starts with an arrow ->, shows the next line that is to be executed. Last but not least, a pdb interactive prompt opens to accept commands from us. Here is a list of commands we will use in order to debug our code:

  • c - Continue the execution until a new breakpoint occurs or the script finished
  • l - List source code around the current line
  • s - Execute the current line and stop at the next possible occasion
  • n - Like s but if the current line is a function call, the whole function is executed and not a single step inside of it
  • p - Evaluate the expression in the current context and print its value (pretty similar but not the same as calling print())
  • q - Quit from the debugger and abort the program being executed (BdbQuit is raised)

Note: You can find a list of all available debugger commands in the pdb documentation [3].

First, we want to know where we are in our code. So let's type l, which shows us right that.

$ python modify.py
> /home/florian/workspace/python/debugging-in-python/modify.py(16)<module>()
-> modify(height, width)
(Pdb) l
 11         z += x + y
 12
 13
 14     height, width = 720, 1080
 15     pdb.set_trace()
 16  -> modify(height, width)
[EOF]
(Pdb)

As you can see, we are right at the end of our file. This is not surprising as we set the only breakpoint right before the last line. The next line that is to be executed is calling modify() with height and width as arguments. We already know that our little file does not contain any bugs, so we have three options now:

  1. inspect the code even further
  2. continue the execution using c
  3. quit the debugger using q

Let's go with the first option and inspect the code further. Entering the n command will execute the whole function call and stop at the very end of the file. Consequently, we can enter s, n or c with all resulting in the successful termination of the script. As we want to see what modify() is doing under the hood, let's enter s instead of n:

(Pdb) s
--Call--
> /home/florian/workspace/python/debugging-in-python/modify.py(8)modify()
-> def modify(x: int, y: int):
(Pdb)

The debugger stopped at the beginning of the function definition. Notice that not only the line number changed in the location output (the line starting with >), but also the scope from <module>() to modify(). Let's make another step using the s command and print the values for y and z:

(Pdb) s
> /home/florian/workspace/python/debugging-in-python/modify.py(10)modify()
-> y, z = z, y
(Pdb) p(y)
1080
(Pdb) p(z)
2000
(Pdb)

Because the execution stops right before swapping both values, z is still 2000 as defined at the beginning of the script and y is still 1080 since we passed the value for width as second argument. Entering s again swaps the values for y and z:

(Pdb) s
> /home/florian/workspace/python/debugging-in-python/modify.py(11)modify()
-> z += x + y
(Pdb) p(y)
2000
(Pdb) p(z)
1080
(Pdb) p(x)
720
(Pdb)

Notice that x is still 720 since we did not touch it. Doing another step assigns the newly calculated value to z:

(Pdb) s
--Return--
> /home/florian/workspace/python/debugging-in-python/modify.py(11)modify()->None
-> z += x + y
(Pdb) p(z)
3800
(Pdb)

As you can see, the line was executed but is still shown as the next line to be executed which is indicated by ->. The reason is that this line contains an implicit return statement as no explicit return statement is given. Printing the value for z reveals that the line was indeed executed and z has a new value. Entering s again jumps out of the function back to the main script and consequently to the end of the file. Thus, doing another step terminates the script.

(Pdb) s
--Return--
> /home/florian/workspace/python/debugging-in-python/modify.py(16)<module>()->None
-> modify(height, width)
(Pdb) s
$

Now, that you know the basics of the Python debugger, I want to show you a slightly different debugging technique: Post-mortem debugging.

Post Mortem Debugging

After a program crashed, you are eager to find out why it did so. pdb allows you to enter post-mortem debugging. Let's take a look at an example to illustrate it.

# div.py


def div(dividend: int = 1, divisor: int = 0) -> float:
    return dividend / divisor

The file div.py contains a single function div() returning the result of dividing two numbers. Both parameters have a default value. Notice that divisor has a default value of 0. Consequently, calling the function without a value for divisor triggers Python to raise a ZeroDivisonError.

Let's open a Python REPL session and see what post-mortem debugging is.

>>> import pdb
>>> import div
>>> div.div()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/florian/workspace/python/debugging-in-python/div.py", line 5, in div
    return numerator / denominator
ZeroDivisionError: division by zero

First, we imported the pdb module as well as our little div module. Next, we called div.div() without any arguments leading to a ZeroDivisionError as expected. Suppose we do not know why a division by zero was made. In this case, entering a post-mortem debug session could help. You can enter post-mortem debugging by calling pdb's pm() function.

Note: pdb.pm() enters post-mortem debugging of the traceback found in sys.last_traceback [6]. You can also enter post-mortem debugging at the traceback of your choice by using pdb.post_mortem(), which accepts exactly one argument, the traceback object.

>>> pdb.pm()
> /home/florian/workspace/python/debugging-in-python/div.py(5)div()
-> return numerator / denominator
(Pdb)

The next line, which is to be executed, is the line raising the exception. We can print the value denominator to confirm that it is a zero causing the ZeroDivisionError.

(Pdb) p(denominator)
0
(Pdb)

Now, we know that denominator is zero resulting in the exception displayed but we do not know why it is zero. Let's use the l command to print some context lines.

(Pdb) l
  1     # div.py
  2
  3
  4     def div(numerator: int = 1, denominator: int = 0) -> float:
  5  ->     return numerator / denominator
[EOF]
(Pdb)

As we can see, denominator has a default value of zero. We called div() without any arguments, which led to the current situation. Consequently, we should either change the default value of denominator or remove the default values entirely if we have access to the source code and can adjust it. Or we could use the information gained from the debug session and call the function with appropriate arguments. The latter is the only option you have if you are not able to modify the source code on your own.

Note: This is a fairly simple example and we already knew the bug as well as the solution for it. However, when dealing with more complex Python code, post-mortem debugging can provide an enormous information gain.

We gained enough information from the debug session, so let's end it by either using the c or q command.

(Pdb) c
>>>

During this section, you learned the basics of the pdb module using two practical examples. But wait, there is more!

Breakpoint() - The New Way to Go

Since version 3.7, Python has a new built-in function called breakpoint() [4]. Simply put: If you do not change anything but use breakpoint() in your code, it is just a convenience function and the same as writing pdb.set_trace() without the need for importing pdb.

However, breakpoint() calls sys.breakpointhook() [5] under the hood and forwards all (keyword) arguments passed to it. Since sys.breakpointhook() is used, every other function can be invoked allowing you to use the debugger of your choice.

I do not want to go deeper at this point because most of the people will not write their own debugger or customize things at this stage. Those people who do this know what to do. However, I wanted to list the breakpoint() function at this point not because it is highly customizable and can do much more than pdb.set_trace(). I wanted to list it as it shows the reader of your code clearly what you want to do: Set a breakpoint!

Summary

Congratulations, you made it through the article! You have learned what a debugger is and what Python comes with out of the box. You had a look at the pdb module and two practical examples showing you the usage of pdb. Furthermore, you met the new built-in breakpoint() function.

I hope you enjoyed reading the article. Feel free to share it with your friends and colleagues! Do you have feedback? I am eager to hear it! You can contact me via the contact form or other resources listed in the contact section.

If you have not already, consider following me on Twitter where I am @DahlitzF or subscribing to my newsletter! Stay curious and keep coding!

References