|Date:||22 April 2009|
I finally snapped last week.
After years of writing verbose and repetitive setup.py files for my Python packages, I am unable to write another. Instead, I have started writing Pyron, a tool that gathers the same information by inspecting a Python package itself. Not only does this mean that I get to stop repeating myself, but that my projects will become much more uniform because package metadata will be represented through common conventions instead of explicit (and repetitive) configuration. Though Pyron is still very primitive, it has already allowed me to reduce simple packages to only a README.txt plus their actual Python source code.
What happened is that I wanted to create a simple Python package full of tools for professional authors working with rst documents, so that they could monitor their word count while writing, and convert their rst files into the proprietary formats used by various publications. But just to start a new Python project required me to create four entire files, and almost as many directories:
./cursive.tools/setup.py ./cursive.tools/cursive/__init__.py ./cursive.tools/cursive/tools/README.txt ./cursive.tools/cursive/tools/__init__.py
The setup.py file itself repeats the project name over, and over, and over again, reminding me of the old Adventure game's “maze of twisty passages, all alike”:
from setuptools import setup setup( name = 'cursive.tools', version = '0.1', description = 'Tools for restructured text files', author = 'Brandon Craig Rhodes', author_email = 'firstname.lastname@example.org', packages = ['cursive.tools', 'cursive'], namespace_packages = ['cursive'], )
The first __init__.py file shown above of course looks like:
import pkg_resources pkg_resources.declare_namespace(__name__)
Meanwhile, my stub README.txt and __init__.py files down in the bottom directory contained just enough information to get me started, whether I wanted to start by writing documentation and tests or get started by writing actual code:
``cursive.tools`` -- Tools for restructured text files ------------------------------------------------------ The routines in this ``cursive.tools`` package are designed for authors. They provide command-line tools that can examine Restructured Text files.
"""Command-line routines for Restructured Text authors.""" __version__ = '0.1'
And, having created these files, I stopped, and stared in horror.
For an entire hour I tried to move on. I tried to start writing actual code and actual documentation. I tried to just ignore the stupidity of what I had just written. Or, in the case of setup.py, what I had just written by cutting and pasting from another project on my hard drive — yes, it's actually become that bad, that we cut-and-paste file contents between Python projects because our boilerplate requires so much repetition while carrying so little information.
But, try though I might, I could not move on to writing code; I was finally defeated. The Python language has done such a wonderful job over the past decade of honing my asthetics and sharpening my senses that I am now unable to use its own standard packaging techquies! This new package would have to wait until I had resolved the problems that sat staring me in the face. Let us review them, one by one.
Obviously, the above arguments hold only for pure-Python packages; when C extensions and other special effects come into play, then excellent reasons arise for a complicated directory structure, sophisticated metadata, and possibly documentation above and beyond that distributed with binary versions of the package. But for normal packages, I am finished with writing and distributing a setup.py by hand.
My new tool for Python package building, Pyron — which, for those keeping score, is my very first bitbucket-hosted project (and I am very much enjoying these first few weeks of using Mercurial, since Guido made the big decision at the end of PyCon last month) — is not yet mature enough to warrant a first release on PyPI. Please check out the development version if you want to take a first look at Pyron. And, yes, Pyron currently has to include a setup.py of its own, which will not disappear until I release the first version and it can become self-hosting!
Please note that Pyron is only for developers! The sdist archives and the eggs produced for a Pyron-powered project are completely standard; the end users and developers installing a module will not be affected by your choice to use Pyron. It simply keeps your project repository cleaner by inferring package metadata on the fly rather than making you maintain a setup.py in version control along with your Python package.
A package developed with pyron only needs two files: README.txt and __init__.py. The two files quoted above will work just fine. These simply need to sit in the same directory, like this:
See? All of the actual meat of the cursive.tools module remains when the files are stored like this, while the while repetition and boilerplate disappears! Check out the Pyron README.txt (or, of course, the same information as formatted in its project page on PyPI) for more details about how it works; here, I will just make three last observations:
Thanks to Pyron, I am now happily working away on my cursive packages, and they should soon see their first releases. I can now sleep at night, knowing that boilerplate and repetition have finally vanished from my development code.