Joshua's Docs - Python Types - Resources, Notes, and Tips

📄 Parent resource: My general Python notes

Resources

Different Python Type-Checkers

List of many different checkers / tools: ethanhs/python-typecheckers

Some of the issues with Python types, compared to say TypeScript or Rust, are:

  • There is a difference between how the types operate at runtime and how they are statically interpreted by a "checker", and historically only some of these rules would be covered by official PEPs
  • There are multiple different type-checkers out there, and they have (historically) not 100% agreed on how they go about parsing, interpreting, and treating types.
  • There is also not really an official Python type-checker; although mypy has strong ties to the official Python team and many people think of it is the de-facto source of truth when it comes to type-checking, the actual typing PEPs (e.g. PEP 484 - Type Hints) generally focus on the code itself and leave interpretation up to the checkers, not advocating for use of any specific checker

Recently (PEP created in September 2023), the official Python team (Python Steering Council) has been moving in an exciting direction and is making headway in standardizing how types are treated across different type-checkers. PEP 729 - Type governing process officially establishes a Python Typing Council (repo), which currently has members from mypy, Microsoft's pyright, Google's pytype, and more! 🎉

To Capitalize or Not

list[str] vs List[str] vs List[Str]

In general, most Python "built-ins", such as list and str, support being used as type annotations and the from typing import List usage is considered deprecated. You can find this documented in the official Python docs.

Typing Functions

To express / represent a function as a type (without actually declaring it), you can use the Callable type:

from typing import Callable
AddTwoIntsFunction = Callable[[int, int], int]

Typing Decorators

Good resources:

Typing While Unpacking / Destructuring

Python supports unpacking, which is a more concise syntax for creating variables out of elements of a list or tuple:

alpha, beta = ['alpha', 'beta']

Unfortunately, if you need to type annotate variables used in unpacking, you have to do it separately above the assignment, not inline:

alpha: Literal['alpha']
beta: Literal['beta']
alpha, beta = ['alpha', 'beta']

Strongly-Typed Dictionaries and Objects

Typed-Dicts

A TypedDict is really just a type-annotation layer over a standard dictionary.

Syntax:

from typing import TypedDict

# Class declaration
class Snack(TypedDict, total=True):
	name: str
	calories: int

apple: Snack = {
	'name': 'Medium Apple',
	'calories': 95
}

# Alternate syntax
Snack = TypedDict('Snack', {'name': str, 'calories': int}, total=True)
apple: Snack = {
	'name': 'Medium Apple',
	'calories': 95
}

Currently, inline / anonymous TypedDicts are not supported, although there is discussion around supporting them

There is are known issue (#10759 and #7981) with mypy and TypedDict.values() returning the wrong types

TypedDict with Generics

Starting with Python 3.11 (but also back-ported in typing_extensions.TypedDict) (PR 27633), you can mix generics with TypedDict, which unlocks some powerful ways to compose interfaces with a minimal amount of code.

PizzaUnits: TypeAlias = Literal['slices', 'pizzas']
SoupUnits: TypeAlias = Literal['bowls', 'cups']

I = TypeVar('I', PizzaUnits, SoupUnits)
class Lunch(TypedDict, Generic[I]):
	unit: I
	price: float

my_lunch: Lunch[PizzaUnits] = {
	'unit': 'bowls',
	#  ^ ERROR: Expression of type "Lunch[SoupUnits]"
	#    cannot be assigned to declared type "Lunch[PizzaUnits]"
	'price': 0.44
}

TypedDicts with Generic Index Signatures

Trying to emulate TypeScript's generic index signatures in Python is not so straightforward (at least at the moment).

It is kind of an all-or-nothing currently, where if every key maps to the same type (or type union), you can do something like:

ImageNameToSize = Dict[str, Tuple[int, int]]
image_map: ImageNameToSize = {
	"vacation.jpg": (243, 424)
}

However, there isn't really (AFAIK) a way to currently strongly type a dict that mixes generic index signatures with some keys that have a more specific type.

PEP 705 seems related, but not likely to solve this.

Nested TypedDict Objects

Currently, the only way to declare a strongly-typed TypedDict object, with strongly-typed nesting, is to declare multiple flat TypedDicts and then combine them into a larger declaration:

class User(TypedDict):
	user_id: int
	name: str
class Page(TypedDict):
	id: int
	author: User
	# ^ Notice how we can't just declare `User` inline

If all the nested objects share the same type, but only the keys differ, you could use a nested regular Dict + Literal for a shorter declaration:

# Dict[Literal["a","b"], MyType]

Finally, if you think generic keys are OK on your nested objects, you can also just use a Dict, which makes the declaration a little easier, but much less type-safe:

class Page(TypedDict):
	id: int
	author: Dict[str, int | str]

Accepting Any TypedDict on a Parameter

"TypedDict" cannot be used in this context - Pylance

Use my_param: Mapping

Type Narrowing

General Type-Narrowing

Here are some general strategies for type-narrowing:

  • To exclude None from being inferred, add branching or early-return code that checks for it
    • E.g. assert my_var is not None
  • To narrow a class instance to a certain class type, you can use isinstance(my_var, MyClass)
  • To narrow a variable to a certain built-in type (in most languages, you might call these primitives), you can also use isinstance(my_var, type), since the built-in types are all classes: int, str, bool, float, list, etc.

Type-Narrowing Lists

Type-narrowing a list generally works better with list comprehension than it does with filter(), at least with the current type-checkers.

Type-Narrowing via TypeGuards

https://adamj.eu/tech/2021/06/09/python-type-hints-how-to-narrow-types-with-typeguard/

Miscellaneous Questions and TypeScript Equivalents

  • NotRequired[T] vs Optional[T]?
    • NotRequired[T] can be used to mark a property as being completely optional
    • Optional[T] means the property is still required, but can be None (type is T | None / Union[T, None])
  • Null or undefined types?
    • Python uses None instead of Null or Undefined.
    • Typing a possibly unset variable?
      • Although you can use possible_string: Union[None, other_type] to represent the type of None | other_type, a shorter way is with Optional[other_type]
    • Checking for None?
      • if my_var is None or even just if not my_var works for checking if variable == None, but not for checking if a variable exists period (this will throw an exception is variable is not defined)
    • How to assert something is not None / narrow to non-None type?
      • assert {MY_VAR} is not None, if {MY_VAR} is not None:
      • If variable is a type that coerces easily to a boolean, then simply assert {MY_VAR} should work, or if {MY_VAR}
  • Declare a string variable as being constant, with shorthand like TS's as const?
    • Doesn't seem to exist. You can use typing.cast to cast, but would still require double-typing
  • Equivalent to TS ReturnType<T>?
  • Equivalent to TS typeof {MY_VAR} (reusing a variable value or inferred type as a new type annotation)/?
  • Equivalent to @ts-ignore?
    • # type: ignore
      • If you get an Invalid "type: ignore" comment error, a possible cause is adding your own text after it. E.g., # type: ignore blah blah
    • For more specific errors, like attribute errors, # type: ignore might not suffice; instead use the more specific type.
      • E.g.: # type: ignore[attr-defined]
  • Equivalent to TS assertions (like myVar as myType)?
  • Why don't if type() checks narrow the type?
    • They can, depending on checker. Try using isinstance() to narrow instead though.
  • Using isinstance() is not working for narrowing
    • Try different syntax approaches.
      • if not isinstance(my_obj, MyClass): instead of if isinstance(my_obj, MyClass) is not True:
  • Equivalent to TS Parameters<T>?
  • Equivalent to uninstantiated class type?
    • Type[ClassType] (typing.Type) or type[] (Python >= 3.9)

Troubleshooting

Looking for troubleshooting or setup specifically for Visual Studio Code?

A word of warning; stubs can (and often are) shipped separately from the actual library; for example, VSCode's pylance extension provides out-of-the-box type stubs for Django, but these might not match the actual version of Django you happen to have installed in your project!

Various issues:

  • Error: Unsupported target for indexed assignment
    • You will often get this when assigning a value to a nested dict property. E.g., my_dict["level_a"]["level_b"] = 2. The root cause here is that mypy et.al. are not perfect at inferring the full type of a dictionary without an explicit annotation
      • Quick solution: Annotate dict as dict - e.g. my_dict: dict = {}
      • Robust solution: Implement a TypedDict and annotate the instance as such
  • TypeError: 'type' object is not subscriptable
    • This usually happens when accidentally indexing into a type or trying to somehow call it:
      • Example: Trying to pass in a type to a generic slot, to a function that does not have a generic slot, like some_fn[Dict[str, Any]](args)
      • Example: Indexing into a type instead of your variable, like dict["my_key"] instead of my_dict["my_key"]
  • Intellisense isn't kicking in when adding elements in an array, dict, etc.
    • Try adding a comma / separator before actually writing out the element. For some reason in VSCode, it won't work unless you do this when adding elements to a list.

Python Types Wishlist / Broken Things

It feels bad to highlight things that are broken in a system that is actively being improved, but there are enough non-obvious "gotchas" and things that hold back my flow while writing strongly-typed Python that it feels worth writing out as a consolidated list so I can more easily keep tabs.

Here are some of the things that are broken, or not yet implemented in Python types (or specific checkers), which I think are important:

Markdown Source Last Updated:
Mon Nov 18 2024 18:43:03 GMT+0000 (Coordinated Universal Time)
Markdown Source Created:
Sat Aug 06 2022 12:32:20 GMT+0000 (Coordinated Universal Time)
© 2024 Joshua Tzucker, Built with Gatsby
Feedback