Archive for the ‘PyEphem’ Category

Eager not to be left behind by the advance of history, I have released PyEphem tonight for Python 3.0! After updating its C-language routines earlier this week, as described in my previous post, and adjusting its Python syntax, I thought that my work was done — until I received a bug report from Reto Schüttel, an enterprising Swiss programmer. He had read my previous post, asked me for the location of PyEphem's Python 3.0 branch, downloaded it with bzr, and already tried it out, not only on his Linux machine, but also on his OS X machine!

While the first twenty revisions of my branch deal with simple Python 3.0 mechanics, I must congratulate Mr. Schüttel for most of the improvements in the subsequent dozen revisions. We not only exchanged emails all day as I produced one revision after another that I needed him to test, but he then joined me on IRC tonight until 3:30am in his time zone as we worked out the last problems.

The issues were all related to localization under OS X. The astronomy library underlying PyEphem used the C-language functions sscanf() and atof() to turn strings into numbers, and it turns out that these functions are very sensitive to locale under the specific combination of OS X and Python 3.0! Because of his locale, the functions wanted a comma instead of a period to separate whole numbers from their decimal fraction (so that π would have to be input like 3,141 rather than like 3.141). They also wanted month name abbreviations to be in German rather than English, ruining my test cases that check planet positions against Naval Observatory tables which use English month abbreviations like Jan, Feb, and Mar with very little regard for how the months would be spelled in German. We are still mystified by the combination of Python version and operation system that were necessary to cause this problem:

Python 2.6 under Linux: tests passed
Python 3.0 under Linux: tests passed
Python 2.6 under OS X: tests passed
Python 3.0 under OS X: broken: sscanf()/atof() change with locale

But the tests worked fine if he put LANG=C on the command line.

To work around this problem, I discovered a wonderful PyOS_ascii_strtod() function in Python's C library that avoided all of the problems that I was having with localization, and so I gradually rewrote the astronomy routines to use that function instead. Fixing the problems with month names was easier; instead of trying to make Python convert month abbreviations by passing the '%b' conversion character to time.strptime(), I simply converted months to integers myself and then passed the integers in with the simpler '%m' format character. It was only late at night that we finally tracked down every routine that was misbehaving.

The final puzzle was how to release my new software. The Python Package Index does not yet allow a project to offer separate source code archives for both the Python 2.x and the 3.0 version of a project. There might be some clever way of storing both versions of the source code in the same .tar.gz file, but the packages I see on PyPI that already support both 2.x and 3.0 contain enough #ifdef statements to convince me that I want to keep my branches separate!

I was saved by a peculiarity of my project. Though it provides the Python package named ephem, the actual project on PyPI has always been named “PyEphem” instead. The product has to have the “Py” in front, you see, to distinguish it both from the original text-screen ephem command and the modern XEphem graphical application, and my first instinct had been to name the project after the product. But as I gained experience with PyPI, it seemed more and more awkward that programmers who wanted their programs to be able to:

import ephem

had to remember to type something different when installing:

easy_install pyephem

Since I had been wanting to switch PyPI names anyway, I suddenly saw my chance! I have now released the new Python 3.0 version of PyEphem under the actual package name ephem and will continue to maintain the 2.x version of the package separately under the old pyephem name. Obviously, this solution only works because of my project's unique circumstances; I have no idea what other projects should do who also want to come out with their new Python 3.0 versions this weekend.

Fans of PyEphem should rest assured that when I develop new features, I will be adding them to both versions of PyEphem for probably at least the next decade. I have absolutely no intention of abandoning or slowing development of the Python 2.x version of the library; I simply wanted the library available to users of the new platform. As always, feel free to email me with bugs, suggestions, and new features — and, enjoy using PyEphem and Python!

With several packages already advertising Python 3.0 compatibility, it seemed high time to look into releasing my PyEphem astronomy package in an edition compatible with the new language. But I hesitated: how difficult is it really, and how many hours of work will it consume, to port a C-language extension module to Python 3.0?

The answer is that, while the necessary changes were surprisingly easy, they took lots of time to figure out because I did not find them documented in any one place. I offer the following notes to assist any other adventurers who want to experiment with porting their extension modules to 3.0. These notes might also suggest useful additions to the official documentation.

But, first, I need to issue three cautions. To develop under 3.0, you may have to forego several Python tools that you probably thought you could no longer do without. The world of 3.0 is a windswept and icy landscape from which the glaciers have just receded, and you will find the stone tools rather primitive when compared to the comforts of civilization that you enjoy under Python 2. To wit:

• I cannot find virtualenv for 3.0, which is a disaster. This means that you have to create a separate Python 3.0 install, built with a different --prefix option to ./configure, for each development environment you want to create on your box.
• I cannot find a version of the setuptools available for 3.0. This means limiting your setup.py instructions to the primitive vocabulary of the distutils package. For example, I find myself unable to run the PyEphem test suite at this late hour because I have been running it for so long with:

  $ python setup.py test

  that I am not sure how to get it running otherwise.
• Should you succeed in porting your extension module, it is not at all clear how to distribute it. I had expected either a new PyPI to spring into being — since every package will need an entirely different version under 3.0 — or for a sophisticated scheme to appear for registering one pyephem.tar.gz as the Python 2 version and another pyephem.tar.gz for 3.0. But while the most recent version of your package can mark itself as 2-compatible or 3-compatible (or both) using classifiers, there is no way to have two “most recent” versions of a package. Are we supposed to start distributing a single tar.gz that includes the source code for both Python series, and that selects the right code by detecting the interpreter version at the top of the setup.py file?

So if you make the effort to port your code right now, you might find that the shiny new version of your module is all dressed up, but has no place to go. If you experiment with the following steps, though, you will at least be ready when an official distribution channel does appear for releasing your package into the wilds of 3.0.

(more...)

I have decided to give my PyEphem astronomy library for Python a public source code repository, an open forum for user questions, and a bug tracker where my users can see the progress of their bug reports out in the open rather than having them scattered across our email inboxes. To accomplish all of this, I simply registered PyEphem with Launchpad, a site built to host software projects that is already used by several projects for which I have great respect.

Because users might become confused now that PyEphem is spread across three web sites — the home page is here at rhodesmill.org, releases are posted over at the Python Package Index, and, again, the development project is now hosted at Launchpad — I have completely redesigned the PyEphem home page with the goal of making the three-site distinction clear, coherent, and easy to navigate. The new home page and documentation are generated by the wonderful Sphinx documentation engine, and I am still thrilled about how pretty my code samples look (check out the one on the PyEphem home page!) now that Sphinx is coloring them in with the renowned Pygments system.

I have simultaneously released a new version of PyEphem that includes the new Sphinx-based documentation, along with several important fixes to the software itself. From now on, rather than cluttering my own blog with every minor version of PyEphem that I might release, fans of the software should visit its News and announcements page on Launchpad and subscribe themselves to its Atom/RSS feed. You will still see the project mentioned here whenever a technical or scientific issue becomes interesting enough for me to write about; but the audience of astronomers and hobbyists who just need to know when the next version is released should not have to wade through my blog to do so!

My users have already begun transferring their questions and problems to Lauchpad, and I look forward to offering much greater accountability through a fully public development process.

On Sunday afternoon I released a new version of PyEphem, which is available from the Python Package Index as version 3.7.2.2! I want to thank the users who spurred its development — in particular, John Duchek of the Astronomical Society of Eastern Missouri encouraged me both to create the new moon-phase functions, and, as I announced several weeks ago, to finally make PyEphem available in binary form for Windows. Thanks, John! PyEphem now also includes a small database of world cities, so that people living near one can simply call ephem.city('Boston') to get their longitude, latitude, and elevation, rather than having to look up the numbers themselves.

Over the years I have received many requests from frustrated Windows users, asking for a Windows-native version of my PyEphem astronomy library for Python. For most Windows users, an attempt to build the extension ends abruptly with the terrible and famous message:

error: Python was built with version 7.1 of Visual Studio, and extensions need to be built with the same version of the compiler, but it isn't installed.

And, as I myself do not have Visual Studio on the small Windows machine that I deign to own for the sake of my photo printer, I have never been able to offer my users much help. But earlier this year, a helpful PyEphem user named Jeff Kowalczyk emailed me a link to Philip von Weitershausen's post “Cheap binary Windows eggs”, which describes a method for building Python extensions using a freely available compiler. (more...)