Article Thumbnail

Introduction to Python's operator-Module

Get an overview of the module and when to use it

Florian Dahlitz
11 min
July 23, 2021

Introduction

When programming in Python, one comes across different situations where working with more or less complex data structures becomes annoying. Fortunately, Python provides a module in its standard library which may help to reduce this feeling: The operator module.

This article aims to give you an introduction to the operator module by having a look at different functions provided by the module paired with hand-selected examples, where to use them.

Let us start by grouping the functions of the operator module, first. In essence, these functions can be divided into two groups:

  1. Functions corresponding to the intrinsic operators of Python.

  2. Functions providing a common interface for attribute and item look-ups.

Intrinsic Operator Functions

We start with the first group, which are the functions related to the intrinsic operators of the Python programming language. Simply put, the operator module provides a function per built-in operator.

For example, the intrinsic add-operator + can be replaced by the operator.add() function:

>>> import operator
>>> 5 + 4
9
>>> operator.add(5, 4)
9

The following table [1] shows the mapping of the intrinsic operators to their functional representation. It includes operators for object comparisons, logical operations, mathematical operations and sequence operations.

Operation Syntax Function
Addition a + b add(a, b)
Concatenation seq1 + seq2 concat(seq1, seq2)
Containment Test obj in seq contains(seq, obj)
Division a / b truediv(a, b)
Division a // b floordiv(a, b)
Bitwise And a & b and_(a, b)
Bitwise Exclusive Or a ^ b xor(a, b)
Bitwise Inversion ~ a invert(a)
Bitwise Or a \| b or_(a, b)
Exponentiation a ** b pow(a, b)
Identity a is b is_(a, b)
Identity a is not b is_not(a, b)
Indexed Assignment obj[k] = v setitem(obj, k, v)
Indexed Deletion del obj[k] delitem(obj, k)
Indexing obj[k] getitem(obj, k)
Left Shift a << b lshift(a, b)
Modulo a % b mod(a, b)
Multiplication a * b mul(a, b)
Matrix Multiplication a @ b matmul(a, b)
Negation (Arithmetic) - a neg(a)
Negation (Logical) not a not_(a)
Positive + a pos(a)
Right Shift a >> b rshift(a, b)
Slice Assignment seq[i:j] = values setitem(seq, slice(i, j), values)
Slice Deletion del seq[i:j] delitem(seq, slice(i, j))
Slicing seq[i:j] getitem(seq, slice(i, j))
String Formatting s % obj mod(s, obj)
Subtraction a - b sub(a, b)
Truth Test obj truth(obj)
Ordering a < b lt(a, b)
Ordering a <= b le(a, b)
Equality a == b eq(a, b)
Difference a != b ne(a, b)
Ordering a >= b ge(a, b)
Ordering a > b gt(a, b)

Even in-place operations are supported, e.g. operator.iadd() for in-place addition. Now comes the obvious question: Why should I import an extra module and use these functions if I can use the intrinsic operators themselves?

I identified two possible use cases, although both are pretty similar and more may exist. The first one uses the fact that Python functions are first-class citizens. This means that you can supply them as an argument to functions, return them, and store them in variables. Let's have a look at a very abstract example.

import operator


def do_something(func):
    ...


if condition:
    fn = operator.add
else:
    fn = operator.mul

do_something(fn)

In the example at hand, we defined a function do_something(), which takes a function as the only parameter and does something with it. For the sake of this example, it is unimportant what is being done. Next, we decide based on a certain condition, which function we need: Either we need a multiplication or an addition. The function can then be supplied to the do_something() function.

Of course, a lambda-function could be supplied as well:

if condition:
    fn = lambda x, y: x + y
else:
    fn = lambda x, y: x * y

... but using the functions from the operator module is more explicit.

The second use case is when using the functools.reduce() functions. In most cases, a list-comprehension is suitable, but if you need to use the functools.reduce() function, functions from the operator module can be helpful.

>>> import functools
>>> import operator
>>> numbers = [1, 2, 3, 4]
>>> functools.reduce(operator.add, numbers)
10

Again, there may be other functions in the standard library or in third-party packages, where supplying a function from the operator module can be considered more readable.

Common Attribute and Item Look-Up Interface

The second group of functions are those providing a common interface for attribute and item look-ups. Although these look-ups can be performed by using the intrinsic operators (see the table above), there are a few advantages of using these curated functions. One of them is that a common interface for different classes or data structures can be established. You will understand the idea when having a look at the examples.

There are three functions falling into this category: attrgetter(), itemgetter(), and methodcaller(). We start by having a look at attrgetter().

Accessing Items by Attribute Name

The idea behind attrgetter() is to create a callable, which fetches the predefined attribute from a given operand. To illustrate that, we create a class Student with three attributes:

  • first_name (str): The student's first name

  • last_name (str): The student's last name

  • student_id (int): The student's personal identification number

Furthermore, a list of three students is created.

# student_class.py
import operator


class Student:
    def __init__(self, first_name: str, last_name: str, student_id: int) -> None:
        self.first_name: str = first_name
        self.last_name: str = last_name
        self.student_id: str = student_id

    def __repr__(self) -> str:
        return f"Student(first_name={self.first_name}, last_name={self.last_name}, student_id={self.student_id})"


students = [
    Student("Albert", "Einstein", 12345),
    Student("Richard", "Feynman", 73855),
    Student("Isaac", "Newton", 39352),
]

The task we are facing is to be able to sort the list of students by their last name and by their student id in combination with the last name. Python's built-in sorted() function accepts a key function (parameter key). In essence, each element of the iterable is passed to the key function and the returned value is used for sorting the elements. We want to have a callable returning the last_name attribute of a given element and a callable, which returns a tuple (student_id, last_name) for a given element, so sorted() can sort the elements based on these values. This can be achieved by utilising the attrgetter() function and assigning the returned callable to variables now acting as functions.

# previous code in student_class.py
get_last_name = operator.attrgetter("last_name")
get_id_last_name = operator.attrgetter("student_id", "last_name")

If we call the get_last_name() callable with an instance of the Student class, the last name will be returned. Simply put, get_last_name(student) is equivalent to student.last_name.

With that being said, we can supply both callables to the built-in sorted() function as key functions.

# previous code in student_class.py
sorted_by_last_name = sorted(students, key=get_last_name)
sorted_by_id_last_name = sorted(students, key=get_id_last_name)

print(sorted_by_last_name)
print(sorted_by_id_last_name)

Running the script via the command line gives us the desired outcome:

$ python student_class.py
[Student(first_name=Albert, last_name=Einstein, student_id=12345), Student(first_name=Richard, last_name=Feynman, student_id=73855), Student(first_name=Isaac, last_name=Newton, student_id=39352)]
[Student(first_name=Albert, last_name=Einstein, student_id=12345), Student(first_name=Isaac, last_name=Newton, student_id=39352), Student(first_name=Richard, last_name=Feynman, student_id=73855)]

Accessing Items by Index

The itemgetter() is somewhat similar to attrgetter(). The difference is that itemgetter() is an equivalent for the index-operator [] and not for accessing attributes. To demonstrate it, we use the previous example and use a tuple of tuples instead of a list of Student instances:

# student_dict.py
import operator

students = (
    ("Albert", "Einstein", 12345),
    ("Richard", "Feynman", 73855),
    ("Isaac", "Newton", 39352),
)

Now, we can create the two callables again, but this time we reference the "attributes" by using their indices:

# previous code in student_dict.py
get_last_name = operator.itemgetter(1)
get_id_last_name = operator.itemgetter(2, 1)

Calling get_last_name(student) is equivalent to student[1]. Again, we supply them as key functions to the built-in sorted() function ...

# previous code in student_dict.py
sorted_by_last_name = sorted(students, key=get_last_name)
sorted_by_id_last_name = sorted(students, key=get_id_last_name)

print(sorted_by_last_name)
print(sorted_by_id_last_name)

... and get the desired output:

$ python student_dict.py
[('Albert', 'Einstein', 12345), ('Richard', 'Feynman', 73855), ('Isaac', 'Newton', 39352)]
[('Albert', 'Einstein', 12345), ('Isaac', 'Newton', 39352), ('Richard', 'Feynman', 73855)]

Calling Methods by Name

Last but not least, let us have a look at the methodcaller() function. It is pretty similar to the previous two functions. This time, the idea is to create a callable, which calls a method on its operand. To show you a practical example, we create three classes first:

  • Plugin: Abstract Base Class (ABC) representing a plugin with an abstract static method run().

  • FirstPlugin: A class inheriting from Plugin and implementing the run() method.

  • SecondPlugin: A class inheriting from Plugin and implementing the run() method, but printing a different value than FirstPlugin.

# plugin.py
import abc
import operator


class Plugin(abc.ABC):
    @abc.abstractstaticmethod
    def run(self):
        pass


class FirstPlugin(Plugin):
    def run(self):
        print("FirstPlugin here")


class SecondPlugin(Plugin):
    def run(self):
        print("SecondPlugin here")

The example at hand could be easily seen as an abstract plugin system. You as the maintainer of a project want to give your users the ability to implement and use custom plugins. Therefore, a plugin base class is provided. The only requirement is that the custom plugin has a run() method accepting no arguments.

If we now want to have a callable, which calls the run() method of a given plugin instance, we can utilise the methodcaller() function to create one:

# previous code in plugin.py
call_run = operator.methodcaller("run")
call_run(FirstPlugin())
call_run(SecondPlugin())

The result of calling it with an instance of FirstPlugin and SecondPlugin is shown below.

$ python plugin.py
FirstPlugin here
SecondPlugin here

The good thing about methodcaller() is that supplying arguments to the method, which is being invoked, is supported, too! So creating a callable ...

callable = operator.methodcaller("run", "foo", bar=1)

... and supplying an instance to it like ...

callable(plugin)

... is the same as:

plugin.run("foo", bar=1)

Summary

Congratulation, you have made it through the article! In this article, you learnt what is inside Python's operator module and got a basic idea of when it is useful. Furthermore, we had a closer look at the three functions attrgetter(), itemgetter(), and methodcaller(), which can assist you in creating common item and attribute look-ups.

I hope you enjoyed reading the article. Make sure to share it with your friends and colleagues. If you have questions or feedback, do not hesitate to reach out to me via one of the channels listed on the contact page. If you have not already, make sure to follow me on Twitter, where I am @DahlitzF and to subscribe to my newsletter, so you won't miss any future announcements!

Stay curious and keep coding!

References