Archive for the ‘Computing’ Category

I would like some advice from Zope and Plone folks about how to create forms that are not only easy for other developers to specialize, but which allow several specializations to be composed together. While I have used zope.formlib and z3c.form before for simple tasks, I have not yet been able to tell whether they support these more advanced kinds of operations.

Some background: I am doing some work on GetPaid for Plone with the generous funding of Derek Richardson who, if his dreams had not carried him away from grad school at the end of the Spring semester, would have tackled this same work as part of the 2009 Google Summer of Code.

The current mechanisms that GetPaid provides for customizing its checkout process are very primitive, and my task is to improve them. That is why I have been thinking about customizing forms.

The August issue of Python Magazine is out, and the cover issue is one of the most exciting that I have had the privilege to publish. Following up on his popular talk at PyCon 2009, professor of computer science Dr. Bill Punch has written an article for us with his colleague Dr. Richard Enbody about replacing C++ with Python in Michigan State University's introductory programming class.

The difficulty they faced was that the second and subsequent classes in Michigan State's curriculum have continued using the C++ language; only the initial course could change to Python. While non-majors taking the introductory course get to learn Python and leave, the computer science majors have to then start the second course without any specific knowledge of C++. How much, skeptics wondered, would it hurt the Python students to be a full semester behind in their C++ knowledge compared to students who took the “old” C++ version of the introductory course?

The circumstances at Michigan State wound up being perfect for making the change to Python a full-fledged, statistically valid experiment to determine how much student grades would be hurt because of the switch. Read the article for the detailed results, but I can report — with great satisfaction — that the Python students proved themselves the equals of their C++ peers, and also that very exciting and unexpected results ensued among the students whom the study was not looking at: the non-majors, for which this was their only programming course, and the seasoned grad students, who rolled up their sleeves and learned Python in order to be able to TA the course. Again, see the article for all of the details!

What else do we have this month? Articles about how easy concurrency becomes with Stackless Python; about documenting your software projects using the powerful Sphinx documentation system; about creating simple graphs with the industry-strength matplotlib plotting library; a guide to the details of hashing algorithms, and how they relate to the Python dictionary implementation; and, finally, the usual thoughts and ponderings from both myself and from Steve Holden of the Python Software Foundation. Come join us!

Like many Unix users, I long ago created a ~/bin/ directory in my home directory and added it to my PATH so that I could supplement the wonderfully rich set of basic Unix commands with some conveniences and shell scripts of my own devising.

The problem, of course, was the chance of collision. Because my shell script names tended to be short and pithy collections of lowercase characters, just like the default system commands, there was no telling when Linux would add a new command that would happen to have the same name as one of mine. This was actually not very likely on, say, a System V Revision 3 workstation in the 1980s, but the trouble became quite a bit more acute when I moved into the world of Debian. Red Hat never really worried me, because they packaged (comparatively) so little software. But Debian today supports a huge number of commands; my modest Ubuntu laptop shows several thousand available:

$ apt-file search -x '^/usr/bin/[^/]*$' | wc -l
21733

The solution was obviously to adjust my command names in such a way that they were still easy to type, but would never be chosen as system command names. For me, “easy to type” means not having to use the shift key, and very few characters turned out to be available, unshifted, on a modern keyboard. The lower-case letters are the very characters used in system commands; brackets, backslashes, the colon, the back-tick, and the single-tick all had a special meaning to the shell; and the slash and dot characters both mean something special in a filename. (The slash divides directory names from filenames, and thus cannot appear in a filename itself, while the dot means “hide this file from normal browsing” if it leads the name, and separates a file from its extension in many other cases.)

There was but one character left: the simple, modest comma.

A quick experiment revealed in a flash that the comma was exactly the character that I had been looking for! Every tool and shell that lay in arm's reach treated the comma as a perfectly normal and unobjectionable character in a filename. By simply prefixing each of my custom commands with a comma, they became completely distinct from system commands and thus free from any chance of a collision.

And, best of all, thanks to the magic of tab-completion, it became very easy to browse my entire collection of commands. When trying to remember which of my commands are available in my ~/bin/ directory on a given system, or when simply trying to remember what some of my commands are called, I simply type a comma followed by tab and my list of commands appears:

$ ,«tab»
,complete-scp        ,go-thpgp            ,range
,complete-ssh        ,gr                  ,svn-store-password
,coreoff             ,hss                 ,umount
,coreon              ,mount-thpgp
,find                ,mount-twt

I heartily recommend this techique to anyone with their own ~/bin/ directory who wants their command names kept clean, tidy, and completely orthogonal to any commands that the future might bring to your system. The approach has worked for me for something like a decade, so you should find it immensely robust. And, finally, it's just plain fun.

I am home from a relaxing vacation to the Midwest, and while I was gone last week my excellent publishing team released the July issue Python Magazine to the world. I am particularly pleased that two of the feature articles in this issue come from important segments of the Python world that we have not heard much from in previous issues.

First, IronPython, the .NET version of Python for Windows, is the topic of Jonathan Hartley's article about acceptance testing. He illustrates that, regardless of the language in which you write your .NET application, you can deploy simple strategies to make your application testable through a Python test harness, and thereby bring Python's strong flexibility as a testing language to bear on a product that you might be writing in another .NET language.

Second, Malthe Borch, a veteran of the Zope community, shares how he has written Chameleon, one of the fastest template language implementations available for Python. By processing each template and turning it into Python bytecode before it is ever used to render a single page, Malthe eliminates a huge amount of redundant processing as that same code is used over and over again. His library is a key ingredient in the new high-efficiency web frameworks appearing in the Zope world. His work might even (fingers crossed) become one of the components that the Plone community uses as they streamline their framework and move towards a lighter and more agile “Plone 4”.

Other technical topics covered are: the Hadoop map-reduce framework; the concept of hash functions, and how they apply to Python; and the new string formatting operator which Guido hopes will replace all of the percent-signs that currently litter our code. Wrap it all up with an editorial by Steve Holden about EuroPython 2009 and an editorial by me, and you have a complete issue! If you do not find Python Magazine sitting on your local newsstand, then I hope you will avail yourself of a subscription and, as always, let me know what topics you would like to see covered in future issues. Enjoy!

Well, drat.

Thanks to more than an hour of work today, I have a pretty list of a few dozen commands that make it easy for a WebFaction account holder to install the powerful lxml Python package for parsing HTML and XML under their hosting account. You can read Ian Bicking's wonderful blog post “lxml: an underappreciated web scraping library” for more information on why you want to be using lxml instead of any of its alternatives.

So, why do I say “drat”?

First, because I just tried out my instructions on another of my WebFaction accounts, and there the extra steps weren't even necessary; this other server of theirs already had lxml's dependencies installed! I suppose, had I been a bit more patient, that this support ticket that I glanced over this morning would have inspired me to ask WebFaction to install the libraries lxml needs on the server where I myself was working. But it felt like some sort of offense against symmetry to rely on something that WebFaction doesn't install everywhere, and I was perhaps just in too big of a hurry. Which, of course, cost more time in the end.

The other reason I say “drat” is because, now that I look at Ian's post again after all these months, I see that he has instructions for making the package install its own dratted copies of the system libraries it needs! Too bad that lxml's own installation instructions omit this crucial piece of information.

How typical, and how predictable. It turns out that I just needed to listen to Ian Bicking more carefully. How often we fail to do that, as individuals and as a Python community. Listen to Ian Bicking, everyone. Listen.

I am very happy with my first weeks of hosting sites at WebFaction. They have taken so much responsibility off of my shoulders. They run nginx in front of my web applications on my behalf. I fill out a simple form, placing Pylons under "/" and WordPress under "/blog", and they write all of the redirect rules to accomplish it. I list the domains and sub-domains on which my site should appear, and they configure the virtual hosting for me.

WebFaction has even finally wrested domain name control from my fingers. Gone are the installations of bind9 that used to be running everywhere, and the smattering of shell scripts with which I wrapped “nsupdate” to publish new host names. Now, I accomplish host name changes in a few moments through the control panel on WebFaction. This gives me more time to do creative things for my customers, and gives me one less daemon that I have to keep upgraded if I want to avoid security problems. (If you want to try them out for yourself, using this WebFaction sign-up link will credit me as the one who recommended them.)

This move to making WebFaction my default choice for hosting new client projects is only the most recent chapter in the very long story of my devolving responsibilities on to other people. The previous steps went something like this:

I am providing a bit of assistance to the wonderful Natural Language Toolkit Project, who have implemented a wide array of language processing algorithms in Python atop a common set of very sleek — and cleanly Pythonic — data structures for representing natural language. If you are at all interested, check out their recent book, Natural Language Processing with Python, which does a great job of showing how the NLTK works at the same time as it explains the computer science concepts of language processing.

The NTLK project wants to support installing their package as a Python egg, so they asked me to tidy up their setup.py file and prepare everything for distribution via the Python Package Index.

As usual, my desire for simple and reproducible behavior when distributing Python packages has run aground on the tangled magic for which setuptools is so justly famous.

The new issue of Python Magazine is out, and it's a technically hefty issue that, while lacking the kind of human-interest “what Python did for me” stories that I often use to balance the articles about programming, does do a really good job of showing the diverse kinds of problems that Python can tackle today.

Before detailing its content, I should apologize that the magazine keeps arriving later and later each month. Alert readers will have noticed that this “June” issue is actually appearing on July 9th. It is late because, in these inexperienced first few months as Editor-in-Chief, I have been treating magazine editing as a serial process, and have not really been starting work on an issue until the previous one is out the door. To fix things, I will start working more like an efficient, modern, pipelined microprocessor, and fetch articles for one issue while a second issue is being edited and yet a third is in layout to be published.

So, watch for a gradually improving schedule over the last half of this year! One reader wrote me to ask whether the late schedule was a sign that the publisher was losing interest in the publication. In fact, it is the person on staff who is most dedicated to Python — myself! — who is at fault, so no lack of interest in the publication is at all implied.

So, what does the “June” issue have to offer?

Plenty!

If programming languages were poets, which poet would the Python language be?

Obviously, Python would be e. e. cummings: the poet for whom whitespace was most truly significant!

The May issue of Python Magazine is now online! Obviously things fell a bit behind schedule this month, as I am writing this five days into the month of June. But with all of the excitement surrounding the publisher's flagship annual conference last month — php|tek 2009 in Chicago — it was certainly better that the magazine slip late than that delays interfere with an event which drew a couple of hundred people to learn more about quality software development.

What can you look forward to reading? This issue worked out as a balance between community news and several sophisticated technical articles on using Python better. The technical articles include: