Archive for the ‘Python’ Category

The August issue of Python Magazine is out, and the cover issue is one of the most exciting that I have had the privilege to publish. Following up on his popular talk at PyCon 2009, professor of computer science Dr. Bill Punch has written an article for us with his colleague Dr. Richard Enbody about replacing C++ with Python in Michigan State University's introductory programming class.

The difficulty they faced was that the second and subsequent classes in Michigan State's curriculum have continued using the C++ language; only the initial course could change to Python. While non-majors taking the introductory course get to learn Python and leave, the computer science majors have to then start the second course without any specific knowledge of C++. How much, skeptics wondered, would it hurt the Python students to be a full semester behind in their C++ knowledge compared to students who took the “old” C++ version of the introductory course?

The circumstances at Michigan State wound up being perfect for making the change to Python a full-fledged, statistically valid experiment to determine how much student grades would be hurt because of the switch. Read the article for the detailed results, but I can report — with great satisfaction — that the Python students proved themselves the equals of their C++ peers, and also that very exciting and unexpected results ensued among the students whom the study was not looking at: the non-majors, for which this was their only programming course, and the seasoned grad students, who rolled up their sleeves and learned Python in order to be able to TA the course. Again, see the article for all of the details!

What else do we have this month? Articles about how easy concurrency becomes with Stackless Python; about documenting your software projects using the powerful Sphinx documentation system; about creating simple graphs with the industry-strength matplotlib plotting library; a guide to the details of hashing algorithms, and how they relate to the Python dictionary implementation; and, finally, the usual thoughts and ponderings from both myself and from Steve Holden of the Python Software Foundation. Come join us!

I am home from a relaxing vacation to the Midwest, and while I was gone last week my excellent publishing team released the July issue Python Magazine to the world. I am particularly pleased that two of the feature articles in this issue come from important segments of the Python world that we have not heard much from in previous issues.

First, IronPython, the .NET version of Python for Windows, is the topic of Jonathan Hartley's article about acceptance testing. He illustrates that, regardless of the language in which you write your .NET application, you can deploy simple strategies to make your application testable through a Python test harness, and thereby bring Python's strong flexibility as a testing language to bear on a product that you might be writing in another .NET language.

Second, Malthe Borch, a veteran of the Zope community, shares how he has written Chameleon, one of the fastest template language implementations available for Python. By processing each template and turning it into Python bytecode before it is ever used to render a single page, Malthe eliminates a huge amount of redundant processing as that same code is used over and over again. His library is a key ingredient in the new high-efficiency web frameworks appearing in the Zope world. His work might even (fingers crossed) become one of the components that the Plone community uses as they streamline their framework and move towards a lighter and more agile “Plone 4”.

Other technical topics covered are: the Hadoop map-reduce framework; the concept of hash functions, and how they apply to Python; and the new string formatting operator which Guido hopes will replace all of the percent-signs that currently litter our code. Wrap it all up with an editorial by Steve Holden about EuroPython 2009 and an editorial by me, and you have a complete issue! If you do not find Python Magazine sitting on your local newsstand, then I hope you will avail yourself of a subscription and, as always, let me know what topics you would like to see covered in future issues. Enjoy!

Well, drat.

Thanks to more than an hour of work today, I have a pretty list of a few dozen commands that make it easy for a WebFaction account holder to install the powerful lxml Python package for parsing HTML and XML under their hosting account. You can read Ian Bicking's wonderful blog post “lxml: an underappreciated web scraping library” for more information on why you want to be using lxml instead of any of its alternatives.

So, why do I say “drat”?

First, because I just tried out my instructions on another of my WebFaction accounts, and there the extra steps weren't even necessary; this other server of theirs already had lxml's dependencies installed! I suppose, had I been a bit more patient, that this support ticket that I glanced over this morning would have inspired me to ask WebFaction to install the libraries lxml needs on the server where I myself was working. But it felt like some sort of offense against symmetry to rely on something that WebFaction doesn't install everywhere, and I was perhaps just in too big of a hurry. Which, of course, cost more time in the end.

The other reason I say “drat” is because, now that I look at Ian's post again after all these months, I see that he has instructions for making the package install its own dratted copies of the system libraries it needs! Too bad that lxml's own installation instructions omit this crucial piece of information.

How typical, and how predictable. It turns out that I just needed to listen to Ian Bicking more carefully. How often we fail to do that, as individuals and as a Python community. Listen to Ian Bicking, everyone. Listen.

I am very happy with my first weeks of hosting sites at WebFaction. They have taken so much responsibility off of my shoulders. They run nginx in front of my web applications on my behalf. I fill out a simple form, placing Pylons under "/" and WordPress under "/blog", and they write all of the redirect rules to accomplish it. I list the domains and sub-domains on which my site should appear, and they configure the virtual hosting for me.

WebFaction has even finally wrested domain name control from my fingers. Gone are the installations of bind9 that used to be running everywhere, and the smattering of shell scripts with which I wrapped “nsupdate” to publish new host names. Now, I accomplish host name changes in a few moments through the control panel on WebFaction. This gives me more time to do creative things for my customers, and gives me one less daemon that I have to keep upgraded if I want to avoid security problems. (If you want to try them out for yourself, using this WebFaction sign-up link will credit me as the one who recommended them.)

This move to making WebFaction my default choice for hosting new client projects is only the most recent chapter in the very long story of my devolving responsibilities on to other people. The previous steps went something like this:

I am providing a bit of assistance to the wonderful Natural Language Toolkit Project, who have implemented a wide array of language processing algorithms in Python atop a common set of very sleek — and cleanly Pythonic — data structures for representing natural language. If you are at all interested, check out their recent book, Natural Language Processing with Python, which does a great job of showing how the NLTK works at the same time as it explains the computer science concepts of language processing.

The NTLK project wants to support installing their package as a Python egg, so they asked me to tidy up their setup.py file and prepare everything for distribution via the Python Package Index.

As usual, my desire for simple and reproducible behavior when distributing Python packages has run aground on the tangled magic for which setuptools is so justly famous.

The new issue of Python Magazine is out, and it's a technically hefty issue that, while lacking the kind of human-interest “what Python did for me” stories that I often use to balance the articles about programming, does do a really good job of showing the diverse kinds of problems that Python can tackle today.

Before detailing its content, I should apologize that the magazine keeps arriving later and later each month. Alert readers will have noticed that this “June” issue is actually appearing on July 9th. It is late because, in these inexperienced first few months as Editor-in-Chief, I have been treating magazine editing as a serial process, and have not really been starting work on an issue until the previous one is out the door. To fix things, I will start working more like an efficient, modern, pipelined microprocessor, and fetch articles for one issue while a second issue is being edited and yet a third is in layout to be published.

So, watch for a gradually improving schedule over the last half of this year! One reader wrote me to ask whether the late schedule was a sign that the publisher was losing interest in the publication. In fact, it is the person on staff who is most dedicated to Python — myself! — who is at fault, so no lack of interest in the publication is at all implied.

So, what does the “June” issue have to offer?

Plenty!

If programming languages were poets, which poet would the Python language be?

Obviously, Python would be e. e. cummings: the poet for whom whitespace was most truly significant!

The May issue of Python Magazine is now online! Obviously things fell a bit behind schedule this month, as I am writing this five days into the month of June. But with all of the excitement surrounding the publisher's flagship annual conference last month — php|tek 2009 in Chicago — it was certainly better that the magazine slip late than that delays interfere with an event which drew a couple of hundred people to learn more about quality software development.

What can you look forward to reading? This issue worked out as a balance between community news and several sophisticated technical articles on using Python better. The technical articles include:

It was back in the early 1990s, if memory serves me, that the Coca-Cola company — it may have been in one of their annual reports — decided to change their perspective. They declared that the soft drink market, of which they held more than 50%, was simply too small to remain their focus. Rather than wondering what marketing campaign might allow them to chip away a few more percentage points from their competitors, they drew attention to the fact that Coca-cola products represented only 2–3% of all fluids consumed, worldwide, and started thinking about how to increase that number. They decided, in other words, that their biggest competitor would no longer be Pepsi, but water.

Now, think about the Python programming language, and try the same shift in perspective. Yes, we are all happy that readers of Linux Journal have selected Python as their Favorite Progrmming Language in this year's “Reader's Choice Awards” and we are pleased when we move up a few more percentage points in something like the TIOBE Programming Community Index. But comparing our market share with that of other programming languages can, too often, fool us into thinking that we are playing a zero-sum game in which our own community can expand only by prying other programmers' fingers off of the languages they currently know and love.

Try something else. Imagine all of the world's people, and ask: what percentage of them use Python? And what might it look like to increase that percentage? You will suddenly think of the grade school down the street, where you could volunteer one afternoon a week teaching Python programming at their computer club. You will realize that the local community college provides a simple IT curriculum, but gives their students no information about getting connected to a local programmer user's group. You will begin to wonder what difference simple Python skills might make in the hands of teenagers, college students, and young families raising children.

You will, in other words, start thinking about how to change the world.

And the reason that I'm excited about the April issue of Python Magazine is that our cover stories focus on Python programmers who, during the recent high-profile United States elections, made this kind of step outward into realizing what Python could do in their communities.

• Mitch Trachtenberg has written our feature article, about how his ballot scanning software helped election volunteers and officials in Humboldt County, California, discover that nearly 200 ballots had not been counted by their Premier Elections Solutions voting systems. Thanks to his efforts, California has now de-certified that particular software from being used in elections in their state.
• We also have an interview with Neal McBurnett, whose ElectionAudits software automated the statistical calculations in Boulder, Colorado, that allowed their election audits to focus on close races where more ballots have to be checked before a winner can be certified with confidence.
• To celebrate the fact that April 22nd was World Plone Day, we feature an Introduction to Plone with instructions for quickly downloading, installing, and customizing this popular Content Management System.
• Our more technical articles describe how to create your own domain-specific language with a parser that gets invoked when you run import on a file with an extension of your own choosing, and how to create a protocol simulation framework that lets you experiment with how packets will traverse a network. And Mark Mruss offers another Welcome to Python article that this time covers Python dictionaries, whose usefulness and elegant design were certainly a big part of my own adoption of the language.
• Finally, Steve Holden shares his perspective as head of the Python Software Foundation, this time reflecting on the wonderful success of PyCon 2009 last month.

I hope you'll take a look at this month's issue, and I hope that you learn as much from reading it as I myself learned from the process of editing. Enjoy!

I finally snapped last week.

After years of writing verbose and repetitive setup.py files for my Python packages, I am unable to write another. Instead, I have started writing Pyron, a tool that gathers the same information by inspecting a Python package itself. Not only does this mean that I get to stop repeating myself, but that my projects will become much more uniform because package metadata will be represented through common conventions instead of explicit (and repetitive) configuration. Though Pyron is still very primitive, it has already allowed me to reduce simple packages to only a README.txt plus their actual Python source code.