Leverage the full potential of type hints

Introduction¶

Python has been and will probably always be a dynamic-typed programming language. Unlike Java or other static-typed languages, Python allows you to assign different value types to a single variable as follows:

>>> x = 5
>>> type(x)
<class 'int'>
>>> x = "Hello World"
>>> type(x)
<class 'str'>

With Python 3.5 and PEP 484 [¹], Python got type hints or type annotations. The aim was not to turn Python into a static-typed language, but instead "opening up Python code to easier static analysis and refactoring, potential runtime type checking, and (perhaps, in some contexts) code generation utilizing type information." [¹]

The overall goal of the article at hand is to give you a general overview of type hints, first. The core of the article is made up of having a closer look at annotated type hints - a feature introduced in Python 3.9 - and how to leverage the full potential of Python's type hints.

To answer the question "What are annotated type hints and why do I need them?", we need to have a look at two things first:

1. Type hints in general
2. Arbitrary metadata annotations

A Quick Look at Annotation Basics¶

Let's start with annotating functions as this is probably the use case most of you have already seen. Suppose we have a function called division(), which takes two parameters (both integers) and returns a floating-point number. A possible implementation looks like this:

# division.py

def division(divident, divisor):
    return divident / divisor


result = division(1, 2)
print(result)  # 0.5

To annotate the function, we can use the following syntax:

# division.py

def division(divident: int, divisor: int) -> float:
    return divident / divisor


result = division(1, 2)
print(result)  # 0.5

After the parameter's identifier, we add a colon followed by the type of the parameter. The return type is specified by adding an arrow -> followed by the type after the parameter list.

Looks good so far! However, a division function only accepting integers is not very useful. Instead, we want to be able to divide floating-point numbers, too. As a result of Python's dynamic-typed nature, we can still call division() with floating-point numbers and don't get an error even though we annotated it with integer types:

# previous code in division.py
result = division(1.5, 2)
print(result)  # 0.75

Even though no runtime exception is raised, a static type checker like mypy will print an error:

$ mypy division.py
division.py:7: error: Argument 1 to "division" has incompatible type "float"; expected "int"
Found 1 error in 1 file (checked 1 source file)

To fix this, we use a typing.Union construct. Let's fix the function annotation first and check, whether mypy is satisfied:

# division.py
from typing import Union


def division(divident: Union[int, float], divisor: Union[int, float]) -> float:
    return divident / divisor


result = division(1.5, 2)
print(result)  # 0.75

$ mypy division.py
Success: no issues found in 1 source file

Great! While the function's annotation is still pretty simple, it can massively grow in complexity. This is especially true if you want to annotate dictionaries with a certain schema. Fortunately, Python allows us to define type aliases. Let's define our own Number type alias, which stands for Union[int, float], and use it for our division() function:

# division.py
from typing import Union

Number = Union[int, float]

def division(divident: Number, divisor: Number) -> float:
    return divident / divisor


result = division(1.5, 2)
print(result)  # 0.75

The same way we annotated the division() function, methods of classes can be annotated. You may find yourself in a situation, where you need to annotate variables as well. Fortunately, variable annotations were introduced in Python 3.6 and specified in PEP 526 [²]. So let us have a quick look at this, too.

In essence, you can use the very same syntax to annotate plain variables in your code:

greet: str = "Hello World"

Annotating variables in loop-constructs is somewhat different. For instance, if you need to annotate a variable in a for-loop, you need to do it before the for-loop:

# loop.py
names: list[str] = ["Peter", "Anne", "Michael", "Beth"]

name: str
for name in names:
    print(name)

That should be sufficient for now.

Arbitrary Metadata¶

In the previous section, we had a look at the basics of type annotations in Python. However, the whole story began a little earlier by introducing arbitrary metadata annotations. In fact, they were the primary reason why annotations were added to Python. PEP 3107 Function Annotations [⁴] defines a way to add arbitrary metadata annotations to functions as follows:

# arbitrary_metadata.py
def money_per_month(money: "in dollar", months: "number of months") -> "dollar per month":
    return money / months


print(money_per_month(3_000, 2))

As you can see, the way the function is annotated is pretty much the same as with type annotations. The difference is that arbitrary strings are used. The idea behind PEP 3107 was to create a single, standard way of specifying this information. Previously, third-party package maintainers came up with their ideas on how to specify function metadata, e.g. through docstrings.

The metadata attached to a function's parameters or return type can not only help users understand the code even better but can be used by third-party libraries. A functions metadata can be access by the __annotations__ attribute:

>>> money_per_month.__annotations__
{'money': 'in dollar', 'months': 'number of months', 'return': 'dollar per month'}

Let's integrate it into our former script and print it in a formatted way:

# arbitrary_metadata.py
import json


def money_per_month(money: "in dollar", months: "number of months") -> "dollar per month":
    return money / months


print(money_per_month(3_000, 2))
print(json.dumps(money_per_month.__annotations__, indent=4))

Executing the script results in:

$ python arbitrary_metadata.py
1500.0
{
    "money": "in dollar",
    "months": "number of months",
    "return": "dollar per month"
}

The issue at this point is that you can only have one of them: Arbitrary function metadata or type annotations. At least until you start using Python 3.9! By concluding this section, we are all set to move on to the core of the article: Annotated type hints.

Annotated Type Hints¶

At this point, we learnt that PEP 3107 added the syntax for arbitrary metadata to Python and PEP 484 provided a standard semantic for the annotations. We had a look at both in the previous two sections. With Python 3.9, annotated type hints approached. They are specified in PEP 593 [³]. The idea of annotated type hints is to bring both - arbitrary function metadata and type annotations - together.

How does it work? With Python 3.9, Annotated was added to the typing module. In essence, it lets you specify the type first, followed by its metadata:

# annotated_type_hints.py
from typing import Annotated


def money_per_month(
    money: Annotated[float, "in dollar"],
    months: Annotated[int, "number of months"]
) -> Annotated[float, "dollar per month"]:
    return money / months

Static type checkers like mypy will only have a look at the type, which is specified first, and completely ignore the metadata following afterwards. To access the type annotations as well as the metadata of the function money_per_month(), we can utilise the function get_type_hints() from the typing module and pass include_extras=True:

# annotated_type_hints.py
from typing import Annotated
from typing import get_type_hints


def money_per_month(
    money: Annotated[float, "in dollar"],
    months: Annotated[int, "number of months"]
) -> Annotated[float, "dollar per month"]:
    return money / months

print(get_type_hints(money_per_month, include_extras=True))

... which results in:

$ python annotated_type_hints.py
{'money': typing.Annotated[float, 'in dollar'], 'months': typing.Annotated[int, 'number of months'], 'return': typing.Annotated[float, 'dollar per month']}

Without include_extras (default), only the types are returned.

We already achieved what we wanted: Bring type annotations and metadata together. What's next? First, we need to refactor the code.

Making use of annotated type hints may add more information to your code, but the code gets more complex at the same time. At this point, defining and using type aliases becomes crucial. Not only does it reduce the amount of code you need to write to use type annotations and metadata, but it prevents silly metadata typos in different places.

We can refactor the annotated_type_hints.py file as follows:

# annotated_type_hints.py
import typing

Dollar = typing.Annotated[float, "in dollar"]
Months = typing.Annotated[int, "number of months"]
Dollar_Per_Month = typing.Annotated[float, "dollar per month"]


def money_per_month(
    money: Dollar,
    months: Months
) -> Dollar_Per_Month:
    return money / months


print(typing.get_type_hints(money_per_month, include_extras=True))

The code looks much cleaner than before. Furthermore, the types can be changed at a central location reducing the maintenance burden. At this point, there is only one question left before concluding this article: Should you always use annotated type hints from now on?

In my humble opinion: Absolutely not! From my perspective, annotated type hints are a great way to add more meaning to your application or especially to libraries. However, while static type checkers exist and are widely used, I am not aware of any widely known and popular tool, which makes use of the metadata. Even type annotations may not always be a good fit (left open for discussions somewhere else).

Maybe also have a look at the article Why I stay away from Python type annotations from Guillaume Pasquet [⁵], where he shares his opinion about type annotations in Python.

To conclude this: I am a big fan of type annotations but I doubt that annotated type hints should be used everywhere. There may be cases, where they are useful and that is exactly why they exist.

Summary¶

Congratulations, you made it through another article! While reading the article, you refreshed your knowledge about type annotations and arbitrary metadata. You learnt the basic evolution of type hints in Python and met annotated type hints, which were introduced in Python 3.9. Last but not least, you read my opinion about type hints in general and annotated type hints in specific.

I hope you enjoyed reading the article. Make sure to share it with your friends and colleagues. If you have not already, follow me on Twitter, where I am @DahlitzF, and subscribe to my newsletter, so you won't miss any future articles. Stay curious and keep coding!

References¶