Month: January 2010

Ugly, Pretty, Special

Another example of the power of young adult fiction.

I visited my family in Indiana over Thanksgiving. On my flight back from Chicago, I was upgraded to first class, seated next to a girl who appeared to be about twelve or thirteen, traveling as an unattended minor. As I sat down she was scribbling furiously in a notebook. She looked up and I smiled. “Hi, there.”

“What’s pithy mean?” she responded, pencil poised to record my answer. My mind went nearly blank; this was not the question I was expecting to answer in the middle of a day of traveling.

“Uh, a short, compact phrase; witty?”

I must have telegraphed more confidence in my answer than I felt: “Thanks,” she said curtly, and went back to her notebook. I pulled out the `Specials <http://en.wikipedia.org/wiki/Specials_(novel)>`_, by David Westerfeld; Dan and Monya had given my `Uglies <http://en.wikipedia.org/wiki/Uglies>`_ and `Pretties <http://en.wikipedia.org/wiki/Pretties>`_ for my birthday just before I left, and after tearing through them, I’d picked up Specials for the trip back. My traveling companion looked up and saw the cover, suddenly becoming animated.

“Oh my god. You’re reading Specials. That’s so awesome.”

“Yeah, I know, it sort of is,” I replied, reverting to a tone more appropriate for gum snapping and borrowing the car than for grown men. “Yeah, like, I read Uglies, and Pretties, and now I’m on Specials; Uglies was great, and Pretties was OK — a bit of a slow down— but Specials is great so far.” Capsule review.

“Totally. I totally agree. Oh my god. So cool.” I seemed to have made a friend, making up for any issues with my vocabulary performance earlier. “Do you have any other authors you recommend?” she asked.

I suggested Octavia Butler, thinking to myself, “kid, this shit is going to blow your mind.” She dutifully wrote it down, checking the spelling as she went.

“I’m writing a story,” she offered. “Sort of science fiction; I mean, it’s set in the future. Well, time has passed from the present; I need the reader to understand that to get the plot.” I continued to nod in solemn agreement and she looked down at her notebook, fingering the pages a moment before snapping her attention back to me.

“Are you a good reader?” How does a thirty-three year old man answer such a question?

“Uh, yeah, I think I’m OK; I like to read,” I answered.

“You know how you can tell a good reader?” She didn’t wait for my reply. “They write down the recommendations from other good readers.” She proceeded to give me some recommendations, which I made sure to write down, not wanting to sully my good reader image.

I finished Specials on the flight home, and it was a fun read. But on my traveling companion’s recommendation, I’ll be checking out the final book in the series, `Extras <http://en.wikipedia.org/wiki/Extras_(novel)>`_, from the library — “it’s totally a let down.” So sayeth the good reader.

date:2010-01-26 10:04:22
wordpress_id:1360
layout:post
slug:ugly-pretty-special-2
comments:
category:my life, reading
tags:fiction, reading, ya

Thoughts on Deploying and Maintaining SMW Applications

In September or October of last year, I received an email from someone who had come across CC Teamspace and was wondering if there was a demo site available they could use to evaluate it. I told them, “No, but I can probably throw one up for you.” A month later I had to email them and say, “Sorry, but I haven’t found the time to do this, and I don’t see that changing.” This is clearly not the message you want to send to possible adopters of your software — “Sorry, even I can’t install it quickly.” Now part of the issue was my own meta/perfectionism: I wanted to figure out a DVCS driven upgrade and maintenance mechanism at the same time. But even when I faced the fact that I didn’t really need to solve both problems at the same time, I quickly became frustrated by the installation process. The XML file I needed to import seemed to contain extraneous pages, and things seemed to have changed between MediaWiki and/or extension versions since the export was created. I kept staring at cryptic errors, struggling to figure out if I had all the dependencies installed. This is not just a documentation problem.

If we think about the application life cycle, there are a three stages a solution to this problem needs to address:[†]_

  1. Installation
  2. Customization
  3. Upgrade

If an extension is created using PHP, users can do all three (and make life considerably easier if they’re a little VCS savvy). But if we’re dealing with an “application” built using Semantic MediaWiki and other SMW Extensions, it’s possible that there’s no PHP at all. If the application lives purely in the wiki, we’re left with XML export/import[‡]_ as the deployment mechanism. With this we get a frustrating release process, Customization support, and a sub-par Installation experience.

The basic problem is that we currently have two deployment mechanisms: full-fledged PHP extensions, and XML dumps. If you’re not writing PHP, you’re stuck with XML export-import, and that’s just not good enough.

A bit of history: When Steren created the initial release of CC Teamspace, he did so by exporting the pages and hand tweaking the XML. This is not a straight-forward, deterministic process that we want to go through every time a bug fix release is needed.

For users of the application, once the import (Installation) is complete (assuming it goes better than my experience), Customization is fairly straight-forward: you edit the pages. When an Upgrade comes along, though, you’re in something of a fix: how do you re-import the pages, retaining the changes you may have made? Until MediaWiki is backed by a DVCS with great merge handling, this is a question we’ll have to answer.

We brainstormed about these issues at the same time we were thinking about Actions. Our initial thoughts were about making the release and installation process easier: how does a developer[◊]_ indicate these pages in my wiki make up my application, and here’s some metadata about it to make life easier.

Semantic Packaging

We brainstormed a solution with the following features:

  1. An “Application“ namespace: just as Forms, Filters, and Templates have their own namespace, an Application namespace would be used to define groups of pages that work together.
  2. Individual Application Pages, each one defining an Application in terms of Components. In our early thinking, a Component could be a Form, a Template, a Filter, or a Category; in the latter case, only the SMW-related aspects of the Category would be included in the Application (ie, not any pages in the Category, on the assumption that they contain instance-specific data).
  3. Application Metadata, such as the version[♦]_, creator, license, etc.

A nice side effect of using a wiki page to collect this information is that we now have a URL we can refer to for Installation. The idea was that a Special page (ie, Special:Install, or Special:Applications) would allow the user to enter the URL of an Application to install. Magical hand waving would happen, the extension dependencies would be checked, and the necessary pages would be installed.

While we didn’t get too far with fleshing out the Upgrade scenario, I think that a good first step would be to simply show the edit diff if the page has changed since it was Installed, and let the user sort it out. It’s not perfect, but it’d be a start.

I’m not sure if this is exactly the right approach to take for packaging these applications. It does effectively invent a new packaging format, which I’m somewhat wary of. At the same time, I like that it seems to utilize the same technologies in use for building these applications; there’s a certain symmetry that seems reassuring. Maybe there are other, obvious solutions I haven’t thought of. If that’s the case, I hope to find them before I clear enough time from the schedule to start hacking on this idea.

date:2010-01-25 21:24:51
wordpress_id:1353
layout:post
slug:thoughts-on-deploying-and-maintaining-smw-applications
comments:
category:cc, development
tags:cc, mediawiki, semantic mediawiki, smw

Ocean Beach Between the Storms

I spent an hour or so Saturday morning at Ocean Beach with my friend Scott, taking some pictures. Scott is a painter and was taking some reference shots. I was playing with the blackbird, fly Richard gave me for Christmas and taking some digital shots as well.

When we got there it had started to sprinkle again, giving us an amazing rainbow that appeared to land on the Cliff House.

Ocean Beach 1

We took a couple photos and went back to the car to wait it out. A few minutes later we had a reprieve from the rain, and we spent the next forty-five minutes walking down the beach.

Ocean Beach 2

The sun to our backs and the clouds over the ocean gave everything an ominous, foreboding appearance. Interestingly, this only made the waves and the water more appealing to me. I love watching waves crashing on rocks, rumbling to the shore; I have never seen the waves at Ocean Beach looking so powerful and so merciless.

Ocean Beach 7

Even with the crashing waves, though, the dogs were still running around and loving it.

Dogs In The Waves

By the time we were close to the Cliff House, it started to rain again. Scott and I hurried off the beach, walking quickly along the sidewalk towards the car. We were soaked when we got there, and the next storm had started.

See the full set of Ocean Beach photos on Flickr; those taken with the blackbird, fly will be uploaded when I get them processed.

date:2010-01-25 09:04:20
wordpress_id:1398
layout:post
slug:ocean-beach-between-the-storms
comments:
category:my life, photo
tags:ocean beach, photos, rain, san francisco, storm

Read: “When You Reach Me” and “First Light”, by Rebecca Stead

I just finished reading First Light, by Rebecca Stead, after reading When You Reach Me earlier this month. I liked When You Reach Me so much that I gave it to James for his birthday, knowing we share a love of science fiction, with a special fondness for young adult science fiction. James has observed that young adult fiction is a joy to read, as it is driven almost entirely by character and story — no messy philosophy or moralistic exposition to get in the way of enjoying the story[1]. When You Reach Me and First Light are both great stories, imaginative and well executed. You can read either trying to divine the “Lesson” for the reader, but that’s optional: they’re just great stories that you can get lost in and enjoy.

When You Reach Me takes place in 1970’s New York City, and tells the story of Miranda and her changing friendships at school. When her best friend, Sal, stops hanging out and she starts receiving mysterious notes, she’s left to figure out what’s going on. When You Reach Me is written with great details that really make the setting and the place real to me. The characters — especially Miranda and her mother — have wonderful dialogue that makes them seem like real, individual people. Stead weaves A Wrinkle In Time through the story, adding to the mystery (it’s clear early on that there’s some relation between the two stories) as well as giving the reader a clue that Miranda is not just any girl, she’s a girl who likes Madeline L’Engle.

When You Reach Me won the Newberry Medal this year, but I actually enjoyed Stead’s first novel, First Light, even more. First Light has one of my favorite structures: multiple, seemingly independent plot lines. You’re sure they’re related some how, and seeing how the author brings them together — as well as how seamlessly — is part of the fun. In the case of First Light, we have the story of Thea, a girl living under the ice, and Peter, a boy from New York City whose father is studying glacial movement and whose mother is prone to “headaches” which take her away from reality for hours or days on end. Just as with When You Reach Me, Stead manages to create a cast of three dimensional, compelling characters, human and canine alike.

[1]For more on the joys of “young adult” literature, see Nick Hornby’s interview in The Atlantic.
date:2010-01-24 21:31:19
wordpress_id:1370
layout:post
slug:read-when-you-reach-me-and-first-light-by-rebecca-stead
comments:
category:reading
tags:fiction, reading, young adult

San Francisco Habitation, Take Two

Last week I took two days off work and, combined with the three day weekend, uprooted my San Francisco existence to date, relocating to the Dogpatch. Not coincidentally, Richard did the same thing, on the same day.

It’s sort of hard to believe it was over two and a half years ago that dad and I drove cross country, spending three days in Motel 6’s and a 16 foot Penske truck along with Maddie. When I was in Indiana for Thanksgiving he said, “Well, I guess you’re coming up on two years in California; how does it feel?” “It feels like I’m coming up on three years,” I said dryly. He was silent as he did the math, realizing that the time has flown, and hopes that life in California is just a phase are just that.

400 Duboce Avenue was a great place for me to land back in 2007. It was close to the Castro, without being in the heart of the Castro. It was convenient to the grocery store. I had plentiful transit options just outside my door. And it turns out lots of people I made friends with were often passing through the neighborhood. But it’s been time to move on for a while.

So I moved to a loft in the Dogpatch on Saturday. I love it, whole heartedly, inside and out. We saw a friend on New Year’s Eve who said, “Man, you should be moving to the Mission, where I am. You just go outside your door and shit’s goin’ on; there’s nothing going on out there.” I simply smiled and said, “Yes, exactly.” Walking Maddie in the evenings, I am delighted at how quiet it is, how non-confrontational. For the first time in two years, I have a kitchen table that isn’t doubling as a desk†. And I have a kitchen with actual counter space. It’s like I’ve graduated from the race car bed to the big boy bed; I sort of feel like an adult again‡.

I made a roast on Monday night for dinner. Richard and I sat at the table — as opposed to on the sofa — with Maddie sitting next to us, patiently waiting to be rewarded for her inherent, obvious goodness. Sitting down to that meal, sitting at that table, I had this feeling of belonging and of home. This is exactly where I want to be right now, and I’m looking forward to seeing San Francisco in a whole new way this year.

† Although now I don’t have a desk, so I guess there was an obvious trade off there. But I do have room for both, now, which is a change.

‡ I think the last time I felt like I had a really adult residence was in my house in Fort Wayne. I don’t think I really appreciated that feeling until this week.

date:2010-01-21 21:29:51
wordpress_id:1374
layout:post
slug:san-francisco-habitation-take-two
comments:
category:my life
tags:400 duboce, dogpatch, move

“Actions” for SMW Applications (Hypothetically)

Talking about AcaWiki has me thinking some more about our experiences over the past couple years with Semantic MediaWiki, particularly about building “applications” with it. I suppose that something like AcaWiki could be considered an application of sorts — I certainly wrote about it as such earlier this week — but in this case I’m talking about applications as reusable, customizable pieces of software that do a little more than just CRUD data.

In 2008 we were using our internal wiki, Teamspace, for a variety of things: employee handbook, job descriptions, staff contact information, and grants. We decided we wanted to do a better job at tracking these grants, specifically the concrete tasks associated with each, things we had committed to do (and which potentially required some reporting). As we iterated on the design of a grant, task, and contact tracking system, we realized that a grant was basically another name for a project, and the Teamspace project tracking system was born.

As we began working with the system, it became obvious we needed to improve the user experience; requiring staff members to look at yet another place for information just wasn’t working. So Steren Giannini, one of our amazing interns, built Semantic Tasks.

Semantic Tasks is a MediaWiki extension, but it’s driven by semantic annotations on task pages. Semantic Tasks’ primary function is sending email reminders. One of the things I really like about Steren’s design is that it works with existing MediaWiki conventions: we annotate Tasks with the assigned (or cc’d) User page, and Semantic Tasks gets the email addresses from the User page.

There were two things we brainstormed but never developed in 2008. I think they’re both still areas of weakness that could be filled to make SMW even more useful as an application platform. The first is something we called Semantic Actions: actions you could take on a page that would change the information stored there.

Consider, for example, marking a task as completed. There are two things you’d like to do to “complete” a task: set the status to complete and record the date it was completed. The thought was that it’d be very convenient to have “close” available as a page action, one which would effect both changes at once without requiring the user to manually edit the page. Our curry-fueled brainstorm was that you could describe these changes using Semantic Mediawiki annotations[1]_. Turtles all the way down, so to speak.

The amount of explaining this idea takes, along with some distance, makes me uncertain that it’s the right approach. I do think that being able to easily write extensions that implement something more than CRUD is important to the story of SMW as a “real” application platform. One thing I that makes me uncertain about this approach is the fear that we are effectively rebuilding Zope 2’s ZClasses, only crappier. ZClasses, for those unfamiliar, were a way to create classes and views through the a web-based interface. A user with administrative rights could author an “application” through the web, getting lots of functionality for “free”. The problem was that once you exhausted ZClasses’ capabilities, you pretty much had to start from scratch when you switched to on disk development. Hence Zope 2’s notorious “Z-shaped learning curve”. I think it’s clear to me now that building actions through the web is going to by necessity expose a limited feature set. The question is whether it’s enough, or if we should encourage people to write [Semantic] Mediawiki extensions that implement the features they need.

Maybe the right approach is simply providing really excellent documentation so that developers can easily retrieve the values the SMW annotations on the pages they care about. You can imagine a skin that exists as a minor patch to Monobook or Vector, which uses a hook to retrieve the installed SMW “actions” for a page and displays them in a consistent manner.

Regardless of the approach taken, if SMW is going to be a platform, there has to be an extensibility story. That story already exists in some form; just look at the extensions already available. Whether the existing story is sufficient is something I’m interested in looking at further.

Next time: Thoughts on Installation and Deployment.

[1]The difference between Semantic Tasks and our hypothetical Semantic Actions is that the latter was concerned solely with making some change to the relevant wiki page.
date:2010-01-07 22:13:41
wordpress_id:1346
layout:post
slug:actions-for-smw-applications-hypothetically
comments:
category:cc, development
tags:cc, mediawiki, semantic mediawiki, smw, teamspace

Read: A Single Man, by Christopher Isherwood

I picked up Christopher Isherwood’s 1964 novel, `A Single Man <http://en.wikipedia.org/wiki/A_Single_Man_(novel)>`_, after Richard and I went and saw the film of the same name, starring Colin Firth and Julianne Moore. The film is beautiful, as you might expect with Tom Ford directing. While the film is largely faithful to the book, the original has its own beauty and nuance that I didn’t see on the screen — not because it’s a poor adaptation, but probably because the book’s nuance, the part I enjoyed the most, is the third person narrator, observing George in the present tense. Isherwood uses the the third person narration to subtly reinforce this feeling of otherness and separation from the world. Telling the story in the present tense, he allows the reader to experience the story as it’s told, unfolding at the pace dictated by the narrator. While I had some idea about the conclusion based on the film, this use of tense and pacing kept me from mentally racing ahead of the narrator.

A Single Man covers one day in the life of George, a gay British man living in southern California in 1962. Not exactly closeted like we understand it today, he’s living a life where he hides in plain site. He goes about his day wearing a mask, measuring his words, paying careful attention to expectations. Jim, his long time partner (although that word is never used, of course), has just died, and people around him either don’t know, or don’t really understand the loss that’s occurred. The book opens with a description of waking up, of the slow return to consciousness, and then the conscious application of the mask to become the George “they” expect. This idea that we have different people, different actors, inside us all is one of the central themes the book explores. It’s a theme I really enjoyed seeing Isherwood play with and explore.

Anyone looking to step into a queer man’s shoes for a day before Stonewall could do far worse than pick up A Single Man.

date:2010-01-06 21:38:40
wordpress_id:1328
layout:post
slug:read-a-single-man-by-christopher-isherwood
comments:
category:reading
tags:gay, reading

We called it “magic”

Just under ten years ago I started working at Canterbury School doing a variety of things. One thing I wound up doing was building the new Intro to Computer curriculum, based on Python. When Vern and I presented our approach at PyCon in 2003, we were asked what advantages we thought Python had over its predecessor in the curriculum, Java. The first answer was always, “Magic; a lack thereof.” There was less boilerplate, fewer incantations, a much shorter list of things you have to wave your hands about and say, “Don’t worry, we’ll talk about this later in the semester. For right now, it’s magic, just do it.” Magic distracts students, and makes them wonder what you’re hiding.

Seeing a comparison between Java and Clojure (albeit one you can read as more about succinctness than clarity), I was reminded that this lack of magic — boilerplate, ceremony, whatever — is still important.

date:2010-01-06 19:04:13
wordpress_id:1317
layout:post
slug:we-called-it-magic
comments:
category:aside, development
tags:magic, python

Caching WSGI Applications to Disk

This morning I pushed the first release of wsgi_cache to the PyPI, laying the groundwork for increasing sanity in our deployment story at CC. wsgi_cache is disk caching middleware for WSGI applications. It’s written with our needs specifically in mind, but it may be useful to others, as well.

The core of Creative Commons’ technical responsibilities are the licenses: the metadata, the deeds, the legalcode, and the chooser. While the license deeds are mostly static and structured in a predictable way, there are some “dynamic” elements; we sometimes add more information to try and clarify the licenses, and volunteers are continuously updating the translations that let us present the deeds in dozens of languages. These are dynamic in a very gross sense: once generated, we can serve the same version of each deed to everyone. But there is an inherent need to generate the deeds dynamically at some point in the pipeline.

Our current toolset includes a script for [re-]generating all or some of the deeds. It does this by [ab]using the Zope test runner machinery to fire up the application and make lots of requests against it, saving the results in the proper directory structure. The result of this is then checked into Subversion for deployment on the web server. This works, but it has a few shortfalls and it’s a pretty blunt instrument. wsgi_cache, along with work Chris Webber is currently doing to make the license engine a better WSGI citizen, aims to streamline this process.

The idea behind wsgi_cache is that you create a disk cache for results, caching only the body of the response. We only cache the body for a simple reason — we want something else, something faster, like Apache or other web server, to serve the request when it’s a cache hit. We’ll use mod_rewrite to send the request to our WSGI application when the requested file doesn’t exist; otherwise it hits the on disk version. And cache “invalidation” becomes as simple as rm (and as fine grained as single resources).

There are some limitation which might make this a poor choice for other applications. Because you’re only caching the response body, it’s impossible to store other header information. This can be a problem if you’re serving up different content types which can’t be inferred from the path (note that we use filenames that look like deed.fr and deed.de, so we tell Apache to override the content type for everything; this works for our particular scenario). Additionally, this approach only makes sense if you have another front end server that can serve up the cached version faster; I doubt that wsgi_cache will win any speed challenges for serving cached versions.

We’re not quite ready to roll it out yet, and I expect we’ll find some things that need to be tweaked, but a test suite with 100% coverage makes that a challenge I’m up for. If you’re interested in taking a look (and adapting it for your own use), you can find the code in Creative Commons’ git repository.

date:2010-01-05 23:37:29
wordpress_id:1309
layout:post
slug:caching-wsgi-applications-to-disk
comments:
category:cc, development
tags:cache, cc, middleware, python, wsgi, wsgi_cache