Python packaging 3

software, python, packaging

I’m trying to figure out how to build Python packages that contains a binary component. There’s ton’s op options out there, scikit-build-core is what I last used. I wanted to repurpose hugo-python-distributions to get a few more commandline utilities up on PyPI, and here’s what I’ve learned:

  1. There is no concept of cross-compilation in Python-land. See this discussion. Agriya very hansomely managed to implement an impressive build-matrix for the hugo-python package, but seeing as Go cross compiles natively, it’s a bit of a pity it’s necessary. cibuildwheel offers some functionality, but that is outside of the Python build, it just preconfigures a platform using QEMU for you. It’s fantastic that it works, but a rather time consuming and cycle-wasting way, given the option that Go already provides. I actually would like to re-use binaries from an existing build system and CI job, so just download them from somewhere, which makes setting up QEMU even more rediculous. If I could tell pip (wheel) for which platform tag it should build, I could tell also which file to fetch. But no, nothing let’s me do that.
  2. I can write Python code that does it, but how do I do that when I just learned to delete setup.py and only use pyproject.toml? I can’t! Building requires code requires setup.py, it’s a simple as that. Well, I could use an external build system such as CMake and drive it using scikit-build-core, but that again is rather overkill than what’s a simple as copying a file based on the target platform.

So, setup.py it is. But I’ll write a new one from scratch for this project and not re-use the somewhat overengineered hugo-python-distributions chain. At least I’ve refreshed my memory on those wheel tags (PEP 425: Python version-Python ABI-platform), that the x and y in manylinux-x-y-arch refer to GLIB version (PEP 600) and that I can set the whole thing by hand if I know better, and when I’m going to just download a binary instead of building it, I should.

What would be nice if setuptools would let you specify what files (and not just deps) to include based on platform, and let you set the platform with a flag so I don’t actually have to be running it.

Serves me right for not simply checking in the binaries into the source tree 😁