Joshua's Docs - Python Notes & WIP Cheatsheet
Light

Resources

What & Link Type
Programming with Mosh: Python Cheatsheet Cheat Sheet
pythoncheatsheet.org Cheat Sheet
Official Docs Official Docs
Drew Learns Python Excellent guide and cheatsheet, especially for JS devs.
Florian GOTO - "Python Essentials for Node.js Developers Intro Guide
PythonTips.com book Cheat Sheet / Quick Reference
RealPython.com Tutorial / Guide Site
Joshua's Docs: Python Types My notes on Python Types
Using VSCode with Python My guide on using VS Code for Python Development

The Module, Package, and Import System

https://docs.python.org/3/reference/import.html https://docs.python.org/3/tutorial/modules.html

Using Packages

The default package manager for Python is called "pip", and lets you install packages from the Python Package Index (PyPi).

For those with NodeJS experience, pip is similar to NPM or Yarn, and PyPi is similar to the NPM Package Repository.

Installing packages with pip is fairly straightforward, and the commands are similar to other package managers:

Action Command
Install from PyPi pip install "PackageName"

You can also use semver:
pip install "PackageName~=1.2.6"
Update package to latest pip install --upgrade PackageName
Install from Requirements File pip install -r requirements.txt
Install from a setup.py file pip install . (or pip install -e . for an editable install)
Install from VCS / Github pip install -e {VCS_Syntax_String} (*)
Install from local tarball pip install {localTarballPath}.tgz
Check installed version pip show {PackageName}
Use --verbose for extra info, like install path

For the full list of commands, please see the pip reference guide.

🚨 Unlike NodeJS package managers, By default, pip installs packages globally. You can --user to scope the install to the user instead, but the best approach is to use virtual environments

Requirements Files

If you are sharing your python script, or even just using it in different environments, you probably don't want to require that its packages (dependencies) be installed one-by-one, manually with pip. A Requirements File is a way to declare all the packages required by your project, down to the specific versions, and which pip can use to provide a repeatable installation experience.

To generate a requirements file, you can either hand-code it, or use the pip freeze command:

python -m pip freeze > requirements.txt

And, then to install from it:

python -m pip install -r requirements.txt

requirements.txt vs setup.py? The usual convention is that requirements.txt is for other developers working on the project / package, whereas setup.py is for consumers of the package.

NodeJS developers: Yes, this is similar to dependencies section of package.json, although pip's package resolution algorithms are not identical to NPM or Yarn.

Working on Nested Packages Locally

Install the package(s) with an editable install:

python3 -m pip install -e {PATH}

Virtual Environments

The default behavior of pip, installing packages globally, is not very optimal once you move beyond a single project; you can easily end up with version conflicts (project A requires lib v1.0, but Project B requires lib v2.0), and other issues.

Luckily, Virtual Environments are a way in Python to have packages scoped to individual project directories, instead of the OS.

In general, if you are using Python 3.3 or newer, it is recommended to use the built-in venv module. Otherwise, the virtualenv tool (as a separate install) is the standard approach. This is a good guide that covers both. And this is a good guide for just venv. You can also find summarized step-by-step instructions for venv below.

Venv Instructions

For convenience, here are the minimal setup steps for venv (assuming you already have Python installed):

  1. Navigate to directory you want mapped / controlled by venv
  2. Create the virtual environment. Convention is to use either env/ or venv/ as the directory.
    • Unix / MacOS: python3 -m venv venv
    • Win: py -m venv venv
    • The folder name you use here usually (depends on shell) shows up in the CLI after activation. E.g. (venv) joshua@desktop-pc >

  3. Activate the virtual environment
    • Unix / MacOS: source venv/bin/activate
    • Win: .\venv\Scripts\activate
  4. Double check that Python is being loaded via the Virtual Environment
    • Unix / MacOS: which python
    • Win: where python
  5. (Optional) Install dependencies, based on requirements.txt
    • python3 -m pip install -r requirements.txt

⚠ REMINDER: Don't forget to activate the environment (step 3)!

💡 Don't forget to exclude the virtual environment directory from Git / Version Control.

When you need to leave the virtual environment, run deactivate. Note that this is not destroying or removing the virtual environment directory - you are just leaving the shell shim / environment. When you want to re-enter the environment, just run the activate script again (step 3 of the above section).

Virtual Environments - Troubleshooting

Venv - Wrong Python Path

I recently found that a venv environment that I had been using for days without issue, suddenly stopped working entirely. Although it would "activate", nothing seemed to function correctly, and calls to which python returned the wrong interpreter path!

The culprit ended up being exactly what this StackOverflow answer covers - I had renamed the parent folder. If you rename the parent directory of a venv, you might need to recreate the environment from scratch!

Venv - WSL Syntax Error

When trying to activate venv on WSL, you might encounter an error like this one, about syntax errors and unexpected tokens:

source ./venv/scripts/activate
	: command not found
	-bash: ./venv/scripts/activate: line 4: syntax error near unexpected token `$'{\r''
	'bash: ./venv/scripts/activate: line 4: `deactivate () {

This is because of a line-ending issue; Python creates activate on windows with CRLF, when it should just be LF (aka \n). To fix, you can patch the file with dos2unix, with tr, or manually in an editor (just do a search and replace for \r\n with \n). I don't believe this can be done in one pass, but I might be wrong (although it would probably require some real shell skills to get as a one-liner).

Arrays and Lists

Array / List Oddness in Python

  • For joining array elements, instead of my_arr.join(separator), Python uses separator.join(my_arr)
  • For filtering array elements, instead of my_arr.filter(fn), Python uses filter(fn, my_arr)
    • Example: list(filter(lambda elem: elem.startswith('b'), ['bear', 'bat', 'cat']))
    • Another pattern is to use comprehension, like: ...mi
  • For sorting arrays, Python uses sorted(my_arr, key=optional_sort_fn)
  • Python lists don't have unshift / append left functionality!
    • Quick solutions:
      • Concatenate: ['a'] + ['b', 'c']
      • Insert: my_arr.insert(0, 'a')

Advanced Indexing and Slicing

For slicing arrays, there is a short syntax that is rather useful:

a[start:stop]  # items start through stop
a[start:]      # items start through the rest of the array
a[:stop]       # items from the beginning through stop
a[:]           # a copy of the whole array
a[-n]          # item n offset from end (a[-1] would be the last element)

Use my_list[0::2] for every even element, my_list[1::2] for every odd. You can use this to extract a list of x coordinates and y coordinates if they are mixed together in a single array.

If you are working numpy arrays, such as NDArray, you have additional advanced indexing and slicing methods available to you. Some of this syntax carries over to assignment as well.

Dictionaries

  • Checking if key exists in dict

    • if 'my_key' in my_dict
  • Dictionary filling shorthand

    const foo = 1;
    const bar = 2;
    const my_dict = {foo, bar}

Unpacking and Destructuring

Unpacking and Destructuring from Lists

For unpacking from an array, list, or tuple use:

drink_sizes = ['12oz', '16oz', '20oz']
tall, grande, venti = drink_sizes

If there are more elements in a list than you want to capture, you can use *my_var to capture leftovers:

tall, *_other_sizes = ['12oz', '16oz', '20oz', '24oz', '32oz']

You can unpack / spread into existing arrays as well:

drink_sizes_oz = ['12oz', '16oz', '20oz']
drink_sizes_all = ['small', 'medium', 'large', *drink_sizes_oz]

Unpacking and Destructuring from Dicts / Objects

Unfortunately, destructuring from dictionaries in Python requires more boilerplate code than in JS, and without shorthand syntax.

Assuming this is our sample dictionary:

# sample dict
drink_orders = {
	'fred': 'Iced Mocha',
	'jennifer': 'Americano'
}

Option A: Use itemgetter or attrgetter:

# Use `attrgetter` if class
fred_order, jennifer_order = itemgetter('fred', 'jennifer')(drink_orders)

Option B: use dict.values()

If you are using Python version 3.6 or above, this method works with any dictionary (as all dictionaries will have guaranteed order), but if you are using 3.5 or below, you will need to use an OrderedDict with this approach.

fred_order, jennifer_order = drink_orders.values()

Formatting

VSCode has an entire docs page devoted to the topic of Python linting.

The PEP8 Style Guide is a very common standard to follow when it comes to Python formatting rules and styles.

Here is a summary of some of the most popular linters and formatters.

💡 For ignoring pep8 errors, you [can add # nopep8 at the end of a line to ignore the error. Other solutions are discussed in this thread. For generally ignoring linting errors, # noqa is accepted by many linters, such as flake8. If you need to ignore issues within a giant triple-quoted string, you can add # noqa at the very end.

Formatting - Weird Edge Cases and Tips

Because Python is a language where whitespace and indenting actually alter how the program runs, there are some... interesting scenarios in which you have to be extra careful about how you format your code.

This is compounded by the fact that the recommended line length limit for pep8 is rather restrictive; just 79 characters!

Here are some special case worth mentioning (WIP):

Long IF Statements / IF with Chaining

When writing a long IF statement, and especially if it uses chaining, you will want to wrap the whole statement in parenthesis. I believe for short statements this would be anti-pythonic, but if you need to wrap due to line length, this will be necessary for parsing:

if (
	students.filter(
		degree: 'Business'
	)
	# You can leave comments mixed between chained methods this way too
	.exclude(
		drop_out_in_progress: True
	)
	.count() > 100
):
	print('Enrollment full!')

Import Statements

from mylib import Alfa, Bravo, Charlie, Delta, Echo, Foxtrot, Golf, Hotel

# To

from mylib import Alpha, Bravo, Charlie, Delta, \
	Echo, Foxtrot, Golf Hotel

Exceptions and Error Handling

Exceptions in Python can and should have distinct types.

There are a bunch of built-in exception types, such as FileNotFoundError and PermissionError, but you can also create your own by subclassing the Exception base class:

For example,

class InvalidEmailError(Exception):
	"""Raise for an invalid email address"""
	pass

if '@' not in email_input:
	raise InvalidEmailError(f'{email_input} is not a valid email')

Testing

Random PyTest Notes

  • How to filter without allowing implicit wildcard at the end?
    • If you want to run a single test file, use pytest {FILEPATH}
    • If you want run a single test function (or method), use pytest "MODULE::MY_TEST" instead of pytest -k "SEARCH_STRING"
  • List matching tests
    • Use --collect-only

How Do I...

  • Avoid key errors / object has no attribute errors?
    • For objects, if hasattr(object, 'my_property'), or use try / catch, or use getattr(object, 'my_property', 'fallback_value')
  • Post or pre-increment?
    • You don't. Use += or -=
  • Run a Python file?
    • python3 -m python_file.py
  • Have an empty indent block?
    • Use pass
    • Within functions, you can also use ... (commonly used in typedefs)
Markdown Source Last Updated:
Sat Sep 10 2022 21:31:50 GMT+0000 (Coordinated Universal Time)
Markdown Source Created:
Wed Nov 11 2020 10:38:21 GMT+0000 (Coordinated Universal Time)
© 2022 Joshua Tzucker, Built with Gatsby
Feedback