Daily Dev Notes 2024/05/28

Finally starting to feel like I'm getting on top of this package publishing business. Yesterday I managed to get a project structure more or less sorted out, which also let me run proper linting, formatting, and tests.

Today I focused on implementing GitHub Actions workflows so that the full test suite runs on commit, and new packages are built and uploaded to PyPi!

The majority of help in getting this set up, came from Simon Willison's excellent blog post: Publish Python packages to PyPI with a python-lib cookiecutter template and GitHub Actions

I didn't follow his instructions to the letter, but reading the post finally made it all click about how to get it working. I did, however, copy his workflows from the datatsette-test package that he linked to from that article. I had to rely on my AI friend to help explain some of the finer points, but I managed to get there in the end.

Some of the key things I learned through this process:

pip install '.[test]' - I had no idea what this was doing at first, but it's very clever. The '.' tells pip to install the current package, which I assume it discovers through the pyproject.toml file. And the '[test]' tells pip to look in the [project.optional-dependencies] section of pyproject.toml and installs the packages listed in the test key, which in Simon's case looks like this: test = ["pytest", "pytest-asyncio"]. I slightly modified this approach for my scenario, but worked great.
GitHub environments - never had a need to use this feature of GitHub before, but the publish workflow recommends using a dedicated environment for building and publishing the package. I'm still not entirely sure exactly what is going on in the GitHub infrastructure that makes this required, but it feels like it makes sense.
Trusted publishers in PyPi - Simon talks about this in his post, but this really feels like magic. You just enter your repo details in the package settings in PyPi, and it lets you push the package from GitHub Actions with no secret tokens or API keys.
Test matrixes - I've had it on my list to investigate tox or Nox to do matrix-based testing, and despite me watching Hynek Schlawack's video on the topic twice, I still don't know which one to choose. And now I can delay that for a bit longer, because the GitHub Action makes it easy to add matrix-based testing like this:

    matrix:
        python-version: ["3.10", "3.11", "3.12"]
        django-version: ["4.2", "5.0"]

Overall, this has been an extremely satisfying process. I've learned lots, and am starting to get a much better handle on this whole process.