When programming in Python, one comes across different situations where working with more or less complex data structures becomes annoying. Fortunately, Python provides a module in its standard library which may help to reduce this feeling: The operator module.
This article aims to give you an introduction to the operator module by having a look at different functions provided by the module paired with hand-selected examples, where to use them.
The code snippets used in this article can be found on GitHub.
Let us start by grouping the functions of the operator module, first. In essence, these functions can be divided into two groups:
Functions corresponding to the intrinsic operators of Python.
Functions providing a common interface for attribute and item look-ups.
We start with the first group, which are the functions related to the intrinsic operators of the Python programming language. Simply put, the operator module provides a function per built-in operator.
For example, the intrinsic add-operator +
can be replaced by the operator.add()
function:
>>> import operator
>>> 5 + 4
9
>>> operator.add(5, 4)
9
The following table [1] shows the mapping of the intrinsic operators to their functional representation. It includes operators for object comparisons, logical operations, mathematical operations and sequence operations.
Operation | Syntax | Function |
---|---|---|
Addition | a + b |
add(a, b) |
Concatenation | seq1 + seq2 |
concat(seq1, seq2) |
Containment Test | obj in seq |
contains(seq, obj) |
Division | a / b |
truediv(a, b) |
Division | a // b |
floordiv(a, b) |
Bitwise And | a & b |
and_(a, b) |
Bitwise Exclusive Or | a ^ b |
xor(a, b) |
Bitwise Inversion | ~ a |
invert(a) |
Bitwise Or | a \| b |
or_(a, b) |
Exponentiation | a ** b |
pow(a, b) |
Identity | a is b |
is_(a, b) |
Identity | a is not b |
is_not(a, b) |
Indexed Assignment | obj[k] = v |
setitem(obj, k, v) |
Indexed Deletion | del obj[k] |
delitem(obj, k) |
Indexing | obj[k] |
getitem(obj, k) |
Left Shift | a << b |
lshift(a, b) |
Modulo | a % b |
mod(a, b) |
Multiplication | a * b |
mul(a, b) |
Matrix Multiplication | a @ b |
matmul(a, b) |
Negation (Arithmetic) | - a |
neg(a) |
Negation (Logical) | not a |
not_(a) |
Positive | + a |
pos(a) |
Right Shift | a >> b |
rshift(a, b) |
Slice Assignment | seq[i:j] = values |
setitem(seq, slice(i, j), values) |
Slice Deletion | del seq[i:j] |
delitem(seq, slice(i, j)) |
Slicing | seq[i:j] |
getitem(seq, slice(i, j)) |
String Formatting | s % obj |
mod(s, obj) |
Subtraction | a - b |
sub(a, b) |
Truth Test | obj |
truth(obj) |
Ordering | a < b |
lt(a, b) |
Ordering | a <= b |
le(a, b) |
Equality | a == b |
eq(a, b) |
Difference | a != b |
ne(a, b) |
Ordering | a >= b |
ge(a, b) |
Ordering | a > b |
gt(a, b) |
Even in-place operations are supported, e.g. operator.iadd()
for in-place addition.
Now comes the obvious question: Why should I import an extra module and use these functions if I can use the intrinsic operators themselves?
I identified two possible use cases, although both are pretty similar and more may exist. The first one uses the fact that Python functions are first-class citizens. This means that you can supply them as an argument to functions, return them, and store them in variables. Let's have a look at a very abstract example.
import operator
def do_something(func):
...
if condition:
fn = operator.add
else:
fn = operator.mul
do_something(fn)
In the example at hand, we defined a function do_something()
, which takes a function as the only parameter and does something with it.
For the sake of this example, it is unimportant what is being done.
Next, we decide based on a certain condition, which function we need: Either we need a multiplication or an addition.
The function can then be supplied to the do_something()
function.
Of course, a lambda-function could be supplied as well:
if condition:
fn = lambda x, y: x + y
else:
fn = lambda x, y: x * y
... but using the functions from the operator module is more explicit.
The second use case is when using the functools.reduce()
functions.
In most cases, a list-comprehension is suitable, but if you need to use the functools.reduce()
function, functions from the operator module can be helpful.
>>> import functools
>>> import operator
>>> numbers = [1, 2, 3, 4]
>>> functools.reduce(operator.add, numbers)
10
Again, there may be other functions in the standard library or in third-party packages, where supplying a function from the operator module can be considered more readable.
The second group of functions are those providing a common interface for attribute and item look-ups. Although these look-ups can be performed by using the intrinsic operators (see the table above), there are a few advantages of using these curated functions. One of them is that a common interface for different classes or data structures can be established. You will understand the idea when having a look at the examples.
There are three functions falling into this category: attrgetter()
, itemgetter()
, and methodcaller()
.
We start by having a look at attrgetter()
.
The idea behind attrgetter()
is to create a callable, which fetches the predefined attribute from a given operand.
To illustrate that, we create a class Student
with three attributes:
first_name
(str): The student's first name
last_name
(str): The student's last name
student_id
(int): The student's personal identification number
Furthermore, a list of three students is created.
# student_class.py
import operator
class Student:
def __init__(self, first_name: str, last_name: str, student_id: int) -> None:
self.first_name: str = first_name
self.last_name: str = last_name
self.student_id: str = student_id
def __repr__(self) -> str:
return f"Student(first_name={self.first_name}, last_name={self.last_name}, student_id={self.student_id})"
students = [
Student("Albert", "Einstein", 12345),
Student("Richard", "Feynman", 73855),
Student("Isaac", "Newton", 39352),
]
The task we are facing is to be able to sort the list of students by their last name and by their student id in combination with the last name.
Python's built-in sorted()
function accepts a key function (parameter key
).
In essence, each element of the iterable is passed to the key function and the returned value is used for sorting the elements.
We want to have a callable returning the last_name
attribute of a given element and a callable, which returns a tuple (student_id, last_name)
for a given element, so sorted()
can sort the elements based on these values.
This can be achieved by utilising the attrgetter()
function and assigning the returned callable to variables now acting as functions.
# previous code in student_class.py
get_last_name = operator.attrgetter("last_name")
get_id_last_name = operator.attrgetter("student_id", "last_name")
If we call the get_last_name()
callable with an instance of the Student
class, the last name will be returned.
Simply put, get_last_name(student)
is equivalent to student.last_name
.
Any class object, which has a last_name
attribute can be supplied to get_last_name()
.
It is not limited to the class Student
.
With that being said, we can supply both callables to the built-in sorted()
function as key functions.
# previous code in student_class.py
sorted_by_last_name = sorted(students, key=get_last_name)
sorted_by_id_last_name = sorted(students, key=get_id_last_name)
print(sorted_by_last_name)
print(sorted_by_id_last_name)
Running the script via the command line gives us the desired outcome:
$ python student_class.py
[Student(first_name=Albert, last_name=Einstein, student_id=12345), Student(first_name=Richard, last_name=Feynman, student_id=73855), Student(first_name=Isaac, last_name=Newton, student_id=39352)]
[Student(first_name=Albert, last_name=Einstein, student_id=12345), Student(first_name=Isaac, last_name=Newton, student_id=39352), Student(first_name=Richard, last_name=Feynman, student_id=73855)]
The itemgetter()
is somewhat similar to attrgetter()
.
The difference is that itemgetter()
is an equivalent for the index-operator []
and not for accessing attributes.
To demonstrate it, we use the previous example and use a tuple of tuples instead of a list of Student
instances:
# student_dict.py
import operator
students = (
("Albert", "Einstein", 12345),
("Richard", "Feynman", 73855),
("Isaac", "Newton", 39352),
)
Now, we can create the two callables again, but this time we reference the "attributes" by using their indices:
# previous code in student_dict.py
get_last_name = operator.itemgetter(1)
get_id_last_name = operator.itemgetter(2, 1)
Calling get_last_name(student)
is equivalent to student[1]
.
Again, we supply them as key functions to the built-in sorted()
function ...
# previous code in student_dict.py
sorted_by_last_name = sorted(students, key=get_last_name)
sorted_by_id_last_name = sorted(students, key=get_id_last_name)
print(sorted_by_last_name)
print(sorted_by_id_last_name)
... and get the desired output:
$ python student_dict.py
[('Albert', 'Einstein', 12345), ('Richard', 'Feynman', 73855), ('Isaac', 'Newton', 39352)]
[('Albert', 'Einstein', 12345), ('Isaac', 'Newton', 39352), ('Richard', 'Feynman', 73855)]
Last but not least, let us have a look at the methodcaller()
function.
It is pretty similar to the previous two functions.
This time, the idea is to create a callable, which calls a method on its operand.
To show you a practical example, we create three classes first:
Plugin
: Abstract Base Class (ABC) representing a plugin with an abstract static method run()
.
FirstPlugin
: A class inheriting from Plugin
and implementing the run()
method.
SecondPlugin
: A class inheriting from Plugin
and implementing the run()
method, but printing a different value than FirstPlugin
.
# plugin.py
import abc
import operator
class Plugin(abc.ABC):
@abc.abstractstaticmethod
def run(self):
pass
class FirstPlugin(Plugin):
def run(self):
print("FirstPlugin here")
class SecondPlugin(Plugin):
def run(self):
print("SecondPlugin here")
The example at hand could be easily seen as an abstract plugin system.
You as the maintainer of a project want to give your users the ability to implement and use custom plugins.
Therefore, a plugin base class is provided.
The only requirement is that the custom plugin has a run()
method accepting no arguments.
If we now want to have a callable, which calls the run()
method of a given plugin instance, we can utilise the methodcaller()
function to create one:
# previous code in plugin.py
call_run = operator.methodcaller("run")
call_run(FirstPlugin())
call_run(SecondPlugin())
The result of calling it with an instance of FirstPlugin
and SecondPlugin
is shown below.
$ python plugin.py
FirstPlugin here
SecondPlugin here
The good thing about methodcaller()
is that supplying arguments to the method, which is being invoked, is supported, too!
So creating a callable ...
callable = operator.methodcaller("run", "foo", bar=1)
... and supplying an instance to it like ...
callable(plugin)
... is the same as:
plugin.run("foo", bar=1)
Congratulation, you have made it through the article!
In this article, you learnt what is inside Python's operator module and got a basic idea of when it is useful.
Furthermore, we had a closer look at the three functions attrgetter()
, itemgetter()
, and methodcaller()
, which can assist you in creating common item and attribute look-ups.
I hope you enjoyed reading the article. Make sure to share it with your friends and colleagues. If you have questions or feedback, do not hesitate to reach out to me via one of the channels listed on the contact page. If you have not already, make sure to follow me on Twitter, where I am @DahlitzF and to subscribe to my newsletter, so you won't miss any future announcements!
Stay curious and keep coding!