Friday was the first full day of talks at PyCon. There were tutorials yesterday, which I did not attend, but which seem to be receiving positive reviews. The morning opened with a keynote by Alexander Limi and Alan Runyan of Plone fame. Not an incredibly technical talk, but interesting to hear how organizations from Oxfam to Burning Man are using Plone to improve their web offerings or collaboration.
After the keynote I went to the Python for Series 60 talk, given by Matt Croydon of unoffical-series-60-python-wiki fame. I didn’t realize that Nokia had pushed their Python implementation out as a SourceForge project, and I was also interested to hear about the existence of libraries for iCal, xpath traversal and jabber. Maybe I’ll finally get that sudoku for series 60 code bundled and shipped.
The second talk of the morning was Python Can Survive In The Enterprise, given by Mike and Dave from AGI. I know Mike from a chance encounter at the Cleveland airport on the way to PyCon 2005, but didn’t know he was presenting until the talk started; glad I caught it. I’m still not sure what it means to be “an industry leading supplier of emoticons” (I paraphrase from their brochure, but not much), but the talk was well done and interesting. The bandwidth and page views they support during peak times (like VD([STRIKEOUT:veneral disease]Valentine’s Day), etc) is incredible. For example, 11 million visits per month on an average day, 35 million during peak months, and 90 million page views on VD itself. And they do it with Python. Of course there was plenty of hand-waving and corporately required vagueness, but very interesting.
The last talk I attended before lunch on Friday was about using PyParsing to build an interactive adventure game engine. Probably not something I’l be using in the near future, but PyParsing does look interesting. So hey, if you need a recursive descent parser…
During lunch Guido gave a history of Python talk, describing what influenced it (besides ABC, which we’ve all heard about). I was particularly interested to hear about the influence of Modula 3 on Python’s method call semantics. Along with GvR’s “State of Python” keynote the following morning, these two talks provided nice bookends on the language’s evolution.
After lunch I went to two back-to-back web framework talks, first on Django, then Turbo Gears. So both contiune to impress, but I’m not certain if its their technical acumen I’m impressed with, or their rapid uptake. What will be interesting, I think, will be to see not only where the frameworks are in a year or 18 months, but where applications written with them today are. I mean, are these tools for long-lived, growing applications, or one-off, deadline driven projects? Not that one of those is necessarily better than the other, they just seem like two different constituencies.
The last talk I attended on the first day of PyCon was on Zanshin, another project being developed at OSAF as part of Chandler. Zanshin is a CalDAV library built on Twisted and used for calendar syncronization in Chandler. I have to admit that I don’t understand the difference between CalDAV and WebDAV, but I really enjoyed this talk. Mostly, I think, because the presenter, Grant Baillie, spent time talking about why some design decisions were made (ie, abstracting the actual connection the way they did) and about the “zen” portion of zanshin. In my notes I have “the temporariness of connectivity” written down and underlined. I know that ccPublisher does not handle disconnects well. It should.
The second day of PyCon started with Guido’s annual State of Python keynote, during which he described some new developments in the Python universe, and things that will be going into Python 2.5. The most interesting new feature to me is the new conditional expression, which will take the format
EXPR1 if COND else EXPR2
Unlike some others, I don’t care about the syntax — it’s clearer than the ternary operator in C, and it reads “right”. And it was only after Guido pointed out a potential bug in the “classic” way to simulate this in Python that I realized how important this is. The classic way to simulate the conditional expression is
(COND and EXPR1) or EXPR2
I don’t know why I didn’t think of this earlier — probably because I haven’t been knowingly been bitten by it yet — but if EXPR1 is False, this falls apart. So it’ll be good to get real support for conditional expressions into the language.
The other interesting bits going with the language are absolute and relative imports and a new with statement. I initially thought this was like Pascal’s with statement — simply a way to reduce the amount of typing you have to do and a way to make your code a little easier to read. Instead it’s much cooler — the object passed to with can have special methods, __enter__ and __exit__ which are called before and after the block, respectively, regardless of any exceptions thrown. So things like lock acquisition/release, atomic transactions and signaling just got a lot easier.
After Guido’s keynote I attended Jeremy Hylton’s talk on the Python Bytecode Compiler. I really regret that I won’t be able to get compiler construction under by belt before graduating in May, so the Python AST stuff holds a certain fascination for me. I’m not sure I follow it all right now, but this is yet another instance where I appreciate my assembly language course work. Weird, I know.
One of the talks done by OSAF staffers at this year’s PyCon was on performing internationalization on Chandler. During this talk Brian Kirsch discussed their PyICU project which wraps the Internation Components for Unicode in a Python SWIG wrapper. It was mentioned that they looked at the zope.i18n library before deciding to go with ICU, and during the Q&A session I asked what it was that zope.i18n didn’t do. It seems like ICU has a much broader scope than just string translation, so it’ll be interesting to see how the Python bindings mature and are integrated into Chandler. In addition to traditional i18n services, Kirsch also touched on converting to and from Unicode in Python. It’s definitely a royal pain in the ass that all strings are not unicode strings, but Kirsch made the point that you should just convert everything to Unicode when it enters your application (decode) and convert back to Bytes when information exists (encode).
After lunch on Saturday I went to a talk about an implementation of the Atom Publishing Protocol for Zope 3. I’m pretty agnostic when it comes to the whole RSS v. Atom thing, but a standard protocol for publishing information does strike me as useful. Unfortunately the system is still under heavy development. On the up side, the presenter was working on it during the Zope Sprint and in talking to him later in the conference, it sounds like he’s made enough progress to finish the initial implementation (there were some problems distinguishing different types of HTTP requests, if I understood correctly).
Just before my talk Ian Bicking gave a standing-room-only talk on Building Pluggable Software with Eggs. Ian managed to provide a good overview of what Eggs are, although most of what he said only really came into focus during the sprint. Or maybe I was just nervous about my talk so I wasn’t really paying attention — either one is possible. Anyway, it was standing room only, the talk scheduled against mine was cancelled, so they scheduled a repeat of it against me at the last minute. Oh well, its not like my talk was particularly spectacular.
So my talk fell during the last slot of the day. I was really proud of myself this year — I actually wrote the paper before doing the slides, and felt like I had put together a compelling “story”. Additionally, I watched some video of myself doing a class presentation the week before the conference in an attempt to figure out how I could improve. It was all in vain. I mean, I don’t think I crashed and burned, but I did rush the material, and as such ran out of steam early. A side effect of rushing through was that I don’t think I ever really made the necessary connection with the audience. Sigh. Hopefully the slides and paper will be useful for others to read through, form their own opinions and ask questions electronically. We’ll see.
After the conference day ended on Saturday Shawn and I went to Fry’s. I heart Fry’s. It’s massive [yes, I know that I sound like a size queen]. I bought a Nokia 770. It’s pretty.
Sunday, the third day of PyCon 2006, definitely felt like the day of rest. People had started to trickle out, and the day had a shorter schedule that the preceeding two days. Which was fine since I was dragging from the night before. The day opened with an anti-keynote: an interview with Bram Cohen of BitTorrent fame. Steve Holden conducted the interview, which was really amusing. Cohen is obviously a hacker in every sense of the word. During the interview he talked about hacking on BitTorrent while intentionally unemployed and living off credit. And about how to hack the credit system: you get dinged when you apply for credit, so simply hoard a large pile of card apps, fill them all out, and then send them out on the same day. It must have worked to some degree since Cohen is now running BitTorrent, Inc (a position he seemed a little ambivalent about — I’m sure his car would have a “I’d rather be hacking” bumper sticker). And Cohen is imminently quotable: “BitTorrent is actually very mathematically lame”, “Python faithfully repoduces the crapitude of POSIX APIs”, and “Niklaus Wirth can bite me.”
After the keynote, Shawn and I went to Ian Bicking’s talk entitled, The Rest of the Web Stack. I’m not exactly sure what I expected, but in the end Ian provided an interesting discussion of the inherent conflict between developers and sys admins: developers want to do cool new things, sys admins want to do the same thing again and again — it’s stable. And of course, the conflict really comes up in smaller organizations where to some extent, the developer is the sysadmin. I’m not sure there were any conclusions — at least none that made it into my notes — but it definitely was interesting to hear someone talking about finding that balance in development efforts.
After Ian’s talk I indulged my current algorithmic obsession and went to Simplying Red-Black Trees. Once the presenter found the room (seriously, there were 3 to choose from), he presented a metaclass he’s used to simplify the implementation of red-black trees at a small performance cost. The basic approach involves identifying attributes which have symmetric behavior, and can therefore be “swapped”. In the case of rb trees, this is the left and right child nodes.
The talk of the morning on Sunday was simply entitled Django How-To, which as it’s title implies was a very straight forward how-to. During the presentation Jacob Kaplan-Moss demonstrated how to build an interactive sudoku web app using Django. Overall I was really impressed with Django — I’m still not sure I love the magic naming of admin classes and such, but its hard to argue with some of the compelling demos. Its definitely something I want to explore further.
After lunch I did the Docutils double-bill, first attending the Docutils Developers Tutorial and then the cryptically titled What is Nabu?. The Docutils Developers Tutorial was less of a tutorial than an overview of the systems available to developers. I’ve been slightly infatuated with docutils lately — rst2s5 is killer — but I’m not sure how I want to extend it (if at all). So it was interesting, but I think like distutils it won’t really gel until I need to extend it somehow.
Nabu, it turns out, is a way of using reStructuredText syntax to semantically (sort of) mark up bits of information. For example, you might have a contact file that identifies how each line relates to the contact itelf. The interesting part of Nabu is that it supports lots of different kinds of information — in fact, I think you extend it in pretty much any way you want, and that it provides a way to collect these bits of information into a common database. This isn’t a system for the “middle class user” but I like seeing people working on marking up information in a way that makes it easier for programs to consume. And maybe Nabu has a future as middle ware — another program for the user interface, presenting an easy to use UI for creation of Nabu-formatted information, and the Nabu server collecting it and making it available for other users.
So I’m more than a little behind on my PyCon wrap up posts, so I’ll start at the end, with the Zope Sprint. This year the sprints moved from pre-conference to post-conference, the idea being that people might have ideas during the conference that they would want to work on at the sprints. I’m not sure that happened, although things said during the conference did spark my interest in certain areas that I was able to work on tangentially during the sprint, so in that respect, I suppose the move was successful. I’m not sure if this was the first year that the sprints were 4 days instead of 2 — I didn’t participate last year, so that change may be two years old. I like it. It really can take 2 days to do enough experimenting to start getting useful things done.
This year (as in the past, save last year), I participated in the Zope sprint. Specifically I was working with the Zope 3 project, as we’re using bits and pieces of it in ccPublisher 2. I spent the 4 days of the sprint working with Python Eggs and Setuptools. Eggs are like Java JAR files, only better — they encapsulate dependency information as well as code. So my task, along with Paul Winkler and others, was to experiment with how Zope 3 could use eggs for packaging and distribution. The idea is sort of two fold: first, lots of pieces of Zope 3 are usable with the “zope” piece — the application server — so it would be good to have an easy way for people to use those pieces easily. The second idea was that because Zope 3 is so flexible, you often don’t need all the different pieces it provides. Maybe you don’t need persistance support (this is probably a bad example, but nothing better comes to mind), so there’s no point in installing and loading it into memory. Since eggs encapsulate dependency information, it’d be nice to be able to do a minimal install, and then use some tool to install additional functionality as you needed it.
Our initial work focused on writing a setup.py for individual projects (zope.interface, zope.deprecation, zope.i18nmessageid) which knew about dependencies and could generate eggs. From there we went on to explore how developers might work on these individual projects. After some work, we came up with a story:
A developer needs to work with Zope page templates. They start out by checking out the pagetemplates project from Zope Subversion onto their local computer. After checking out the project, they have a directory containing setup.py, a src directory, and other documentation. They run a [waving hands] developer script, which creates a local sandbox, and downloads all of zope.pagetemplate’s dependencies into it. These dependencies are downloaded from the Cheeseshop or the GZRITS. After some work on page templates, a developer realizes they need to work on a dependency. So they check out that dependency into the same zope.pagetemplates working directory, and run it’s setup.py develop. Develop is a distutils target provided by setuptools which gets the dependencies for a project, and also installs a special “link egg” into your local Python path. You do your work, run your tests, run the dependency tests, and when everything passes, you commit.
And we generally got that story working. In fact, I think I did two implementations (it sort of runs together) — one as a standalone “develop.py” script, and the other as custom distutils commands. That was one of the unexpected benefits of the sprint: I now feel like I can confidently hack things I need into distutils without considering the entire process so much black magic.
The last day of the sprint also was one of the most productive for me. Jim and I were discussing how to proceed with eggs, and decided that if it was easy, a tool that read zpkg dependency information and generated eggs would be a good transitional tool. So zpkgegg was born. I did most of the work during that last day, and then worked on polishing it on the flight home and wrapped up on Saturday. I’m pretty pleased with the way it turned out, even if it is just a transitional tool.
In my mind the sprint was well worth the time I spent there and the money CC spent on me attending it. Zope 3 and Eggs are both technologies that are central to ccPublisher’s future: Zope 3 because it provides so much in the way of infrastructure, and Eggs because they’re the most logical choice for an extension packaging mechanism. And the knowledge gained at the sprint has already paid off — in about an hour yesterday I created a branch of ccPublisher and hacked it to use eggs instead of svn:externals for the Zope dependencies.
I arrived in [STRIKEOUT:Dallas]Addison, TX Thursday night for PyCon 2006. As I’ve mentioned before, this is the first year PyCon moves from Washington, DC, and by and large its a good change. Addison is a lovely town, if you’re into strip malls and big hair. I kid, but not much. Having the conference at a hotel does have the advantage that you can easily drop things back at your room, and we have lots more space for tracks. And the food isn’t bad, either. OK, the wireless still sucks, and it’s not free in the rooms, but that’s no different than before. I had intended to blog the event as it happened, but you know what they say about the best laid plans. So here are my thoughts on PyCon 2006 as a whole; a series of shorter posts will follow as I work through my notes on particular talks.
So the most obvious change was the con moving to Texas. I have mixed feelings about Texas. You know, ever since spending time in Amarillo, I just don’t feel too fondly for it. But I put that aside; PyCon is usually the most productive and worthwhile conference I attend each year. An unexpected difference related to the change of venue is the change of atmosphere around the conference. Not the conference itself, but its surroundings. In Washington, there was a small grocery in the basement, a food court off the lobby and lots of diversity in walking distance. In Addison, there’s still food with walking distance, but its a further distance, and the hotel generally feels a little more separated from the surrounding community. I’m not sure this is a bad thing — in Washington when the day’s events ended lots of people scattered. In Addison there seemed to be lots of group dinners and late night hacking, which is cool. And really, if resturants where distance n from the conference in Washington, they’re no more than 3n here, which as we all know is the same O().
I was also generally impressed with the quality of talks this year. In past years there have always been stand-out talks, but there were usually holes in the schedule where nothing on it really appealed to me. There were still a couple of those this year, but noticeably fewer. Finally, I give two-thumbs up to the food; no wrap in sight. Hooray!
So today begin the sprints. I’m working on the Zope3 sprint again, after skipping it last year (why was that again?) so I’m looking forward to getting back into that mode. More on PyCon to come over the next few days.
So registration is open for PyCon 2006, taking place in lovely greater Dallas late February, 2006. Here’s a few links for the interested:
Once again I’ll be presenting this year. My talk, “Extensible Desktop Applications: Abusing the Zope 3 Project”, will cover how we’re using pieces of the Zope 3 project in our desktop apps at Creative Commons. Should be a good time.