July 2013 – Nathan R. Yergler

Destroyed 0002

I’m blogging my way through Gary Bernhardt’s excellent Destroy All Software series of screencasts. If you write software, you should probably go buy it today.

Episode 0002 of Destroy All Software talks about nil in Ruby. I’m not a Rubyist. I may be someday, but I’m not today, so I just imagined he was talking about None in Python with weird syntax. This episode is really how returning a nil value can lead to exceptions that are miles away from where they actually originated. Gary demonstrates this with a little Rails app, and I found myself nodding along: I see this with some frequency in the Eventbrite codebase, where a domain model’s property is set to None, and at a later point other code tries to call a method on that value.

You can, of course, write your own property descriptor (in Python) that checks for None and raises an exception when that value is set. At least then the error is localized to when it’s really being set to (or returning) None. But what you really want is to avoid the error altogether. Gary shows a couple ways to potentially do that, including inverting the relationship between domain models, and introducing a new model instead of just setting a property on an existing one.

author:	Nathan Yergler
category:	destroyed
tags:	til
comments:

Destroyed 0001

Some people blog their way through Knuth or SICP. My attention span is somewhat shorter lately. I’ve recently begun watching Gary Bernhardt’s excellent Destroy All Software screencasts, and I thought it’d be fun to blog my way through it with a series of short posts on what I learned from each episode. I’ve watched a few as I start this, and I think that if you write software and care about writing good software, you should probably go buy DAS now.

I watched Episode 1 on my way to OSCON about a week ago. In it Gary works through building a small bash script to calculate some statistics on a git repository (for example, how many lines of code there were at given points in time). The git plumbing bits were pretty interesting, but it was the actual process that was really educational.

One of the first things he also does is map a key to save and then run his script. I almost found myself coveting Vim for a moment, because it seems obvious now that having an immediate feedback loop is actually superior to switching between Emacs and a terminal.

As Gary builds out the script, he points out a few things, like using set -e, and “always quote your arguments”. (It makes a missing argument fallback to an empty string, which programs like grep are perfectly happy with.) That sort of casual, fingertip knowledge is a joy to watch. I guess I haven’t written enough in bash to know better than to check my exit codes manually for things. set -e is obviously better. Way better.

And have you ever considered that bash control structures like while and for have a stdin and stdout? They do. It seemed obvious once I saw him do it, and when I think about the way bash works, it makes sense in a consistency sort of way. But until now I’d never considered piping the output of, say, grep to a control structure.

Watching DAS S1E1 I learned a few things about shell scripting that seem really fundamental, which I wish I’d have known about for, well, years. I also realized that I have this weird mix of git knowledge: I understand that it’s a directed acyclic graph and a bunch of the underlying structures. I also am proficient at using magit to manipulate a repository within Emacs. The git porcelain? Not so much.

Finally, I thought it was interesting to see and listen to Gary refactoring a bash script using some of the same principles that I use when looking at Python code. Specifically, wanting to make code easy to read, not just execute.

author:	Nathan Yergler
category:	destroyed
tags:	til
comments:

Emacs & Jedi

Brandon Rhodes delivered the keynote at PyOhio yesterday[1]. He talked about sine qua nons: features that without which, a language is nothing to him. One of the things he mentioned was Jedi, a framework for extending editors with Python autocompletion, documentation lookup, and source navigation. I say “editors”, vaguely, because Jedi consists of a Python server that the editor communicates with via an editor specific plugin. I’d seen Jedi before, but hadn’t managed to get it working with Emacs. After hearing Brandon speak of it so glowingly, I decided to give it another try. The actual installation was easy: using the master branch of el-get, the recipe installed the Jedi Emacs plugin and its dependencies seamlessly. And it seemed to just work for the standard library.

And once I’d enabled it for python-mode, I was indeed able to autocomplete things from the standard library, and jump to the source of members implemented in Python. But I found that I wasn’t able to navigate to the third party dependencies in my project, and eventually figured out there were three cases I needed to address.

Many things I work on use virtualenv. Jedi supports virtual environments if the VIRTUAL_ENV environment variable is set, but I tend to keep Emacs running and switch between several different projects, each with their own environment.
Some of my projects also use buildout. When I’m using buildout for a project, the dependencies end up in an eggs sub-directory, which Jedi (as far as I know) doesn’t actually know about.
Finally, the setup we use at Eventbrite requires some special handling, as well: we store our source checkouts on an encrypted disk image, which is then mounted into a Vagrant virtual machine, where the actual virtual environment lives. Since the virtual environment isn’t on the same “machine” that we’re editing on, I need to tell Jedi explicitly what directories to find source in.

The Jedi documentation includes an advanced example of customizing the server arguments on per buffer. It assumes static arguments, but it seemed like a solution was possible. I spent a couple hours this afternoon working on my Emacs Lisp skills to make Jedi work in all three of my cases.

kenobi.el

kenobi.el (gist) is the result, and it does a few things:

Fires hooks for each mode after the file or directory local variables have been set up. I found a StackOverflow post that confirmed what I had observed: any file or directory local variables weren’t set when the normal python-mode hook was fired. The mode-specific hooks are chained off of hack-local-variables-hook, which is fired after the local variables have been resolved.
Walks up the directory hierarchy from the buffer file, looking for ./bin/activate at each level. If it finds one, it assumes this is the virtual env, and adds it to the list of virtual envs Jedi will look at.
Walks up the directory hierarchy looking for an ./eggs/*.egg sub-directory at each level. If it finds one, it adds each of those .egg subdirectories to the sys.path Jedi will look at. This allows Jedi to work when you’re editing files in buildout-based projects.
Looks to see if the aditional_paths variable has been set as a list of other paths to add.

The first three bits are sort of implementation details: you can usually just ignore them, and Jedi will just work. The final, though, needs a little explanation.

As I mentioned above, Eventbrite stores the source code on a disk image, which is mounted into a virtual machine where the actual virtualenv lives. That means I need to add specific paths to sys.path when I open a source file in that disk image. To get that to work, I create a .dir-locals.el in the root of the source tree, something like:

(
 (nil . ((additional_paths . (
          "/Volumes/eb_home/work/src"
          "/Volumes/eb_home/work/deps"
          ))))
)

I’m sure that my Emacs Lisp could be improved upon, but it felt pretty good to figure out enough to integration Jedi into the way I use Emacs. I haven’t worked with Jedi extensively, but so far it seems to work pretty well. The autocomplete features seem to be minimally invasive, and the show docstring and jump to definition both work great.

[1]	The fact that the video is up less than 36 hours after the keynote is a testament to how great Next Day Video is. They do amazing work at Python (and other) conferences and make it possible to enjoy the hallway track without worrying about missing a presentation.

author:	Nathan Yergler
category:	emacs
tags:	python, virtualenv, buildout
comments:

Draft Work: “Fast Pass”

3” x 5” two plate linocut print

Last week when I was printing my line study, I had a couple extra hours in the studio. I’d previously drawn the plates for a Fast Pass double-plate print, so I quickly carved it as a fun distraction. The Fast Pass was the SFMTA monthly pass card when I moved to San Francisco in 2007. It’s since been supplanted by Clipper, but most of my friends have some emotional attachment to the Fast Pass. You had to get a new one every month, and the colors changed each time, often appearing almost seasonable. I have a collection of the Fast Passes that passed through my hands, and it seems like I’m not alone.

As a draft print that took about 90 minutes to carve, I’m happy with the result. I’m particularly happy with how the MUNI logos came out. This is one of the first prints I’ve done with text on it, so I think the next thing I’d like to work on is cleaning up the text carving a bit.

author:	Nathan Yergler
category:	printmaking
tags:	linocut, multiplate, text, postive-negative
comments:

Effective Django at OSCON Post-Mortem

I’ve been on the road a little more than a week now, back to back conferences. On Tuesday I presented my Effective Django tutorial at OSCON. I’ve recently updated it to cover integrating static assets with your project, and re-organized some of the view information. The biggest challenge with presenting a tutorial like that is figuring out how fast (or slow) to go. I’ve practiced the tutorial and use Django daily, so of course I’m able to type and diagnose what’s going on more quickly. At the break on Tuesday a few people asked me to slow down, but as I walked around the room, two others told me (quite kindly) that they wouldn’t mind if I sped up. This was a little worse than at PyCon, I think, primarily because I didn’t have friends and colleagues assisting me. At PyCon they were able to wander the room and help backfill support for those who were moving a little slower.

I think the next time I deliver Effective Django — or any tutorial — I’ll probably try to mitigate with a few different ways:

Make sure I have assistants. I actually had one planned for OSCON, but he unfortunately fell ill. Next time I’ll try to have more than one scheduled (I had four at PyCon, three is probably fine).
Provide more guidance in the synopsis. If I’m going to assume no prior Django knowledge, I should probably say something like “We’ll start from zero and build a Django application together,” rather than just relying on the slightly ambiguous Novice tag in the program.
Provide some clear stopping points. I noticed that some attendees stayed with it for the first half, or first three quarters, but disengaged before the end. I tried to structure the tutorial so that you can stop at any time and still have learned something, but I could be more explicit about that. “And now we know how to write views that use the ORM,” rather than “We’ll see in a moment how to wire that into the …”
Make sure my handouts are ready to go. At OSCON I didn’t realize they were sitting in cardboard boxes off to the side until half way through. I noticed that in the second half, a couple of the people who had been struggling previously were keeping up slightly better once they had another source to refer to.

The people I spoke to afterward uniformly said they learned something (whether it was as much as they’d hoped is another question, I suppose), so this feels like an overall success.

author:	Nathan Yergler
category:	talks
tags:	oscon, pyohio, effective django, python3
comments:

Coffee Cup line study

4” x 5” linocut print

I wanted to practice using lines to describe, rather than outline, shapes and surfaces, so I took a picture of a coffee cup on a sunny day and decided to try and make a print from it. I worked small and (relatively) simple to avoid investing too much time in what is (effectively) a practice piece.

It was pretty instructive to carve this in an afternon, and then print it on Wednesday. Because it was small project, I was able to remember what I expected when I was carving, and compare that to what came out. There were a few things that came out as expected, and a few that didn’t. That was sort of the point.

author:	Nathan Yergler
category:	printmaking
tags:	linocut, study, practice
comments:

New Work: “Candlestick Point”

8” x 10” linocut print, printed on Magnani Pescia

A dear friend sent me a photograph she took from Candlestick Point shortly after I made a print of a canal in Amsterdam. She noted that I’d been working with images of water, and thought of me when she looked out over the bay. I’ve been thinking about the image for a while, and started working on the actual print a couple months ago. I was finally ready to print on Wednesday.

“Candlestick Point” plate, midway through carving

In this case I made a drawing from the photograph, transferred it the lino plate, and then starting thinking about how to actually carve things away. I was about 90% of the way done when I scanned the plate, as an experiement. I think it’s pretty interesting to look at, too. The rocks, in particular, are something new for me. It’s taken me a while to ease into using parallel lines like that to indicate shading and shape. It’s a technique I’ve seen others use very effectively, and I’m excited to develop it for my own work.

author:	Nathan Yergler
category:	printmaking
tags:	linocut
comments:

“Effective Django” at OSCON

I’m going to be presenting my introductory Django tutorial, Effective Django at OSCON later this month. If you’re going to be at OSCON and haven’t selected your tutorials yet, or just think a trip to Portland, Oregon sounds nifty, there’s still time to sign up. You can find the details on the OSCON tutorial page.

In preparation for that I’ve been continuing to work on the content. I presented the tutorial to some of the Eventbrite engineering team a couple weeks ago, and thier feedback was very useful. In response, I’ve made a few changes. Specifically, I split up the Views chapter with a brief interlude on static assets and template inheritance. It’s something that I didn’t cover the first time around, but based on the questions, I think some guidance is useful.

The revisions for OSCON also include updating the sample code repository that goes with the tutorial. I developed a tool, Tut, to help manage these stacked branches, and while making changes to early parts of the tutorial code, I realized it still requires significant work to really be a good workflow tool. One of the most important requirements for Tut is the ability to manage an ordered series of “checkpoints” and move between them.

When I started working on the sample code this time around, I was on a new laptop, so I had to start from a fresh clone. This was revealing and frustrating. I discovered Tut assumed all the branches were already local, which they obviously aren’t with a fresh clone. Worse, the git magic I was trying to use to get the branch list in the “right” order was pretty fragile, and broke when I tried to lean on it at all.

This screenshot shows a common editing case. My intention is to manage each step, or checkpoint, in the tutorial as a branch. Each step builds on the previous one, so if I make a change to something early in the tutorial, I just need to merge the branches “forward” until I get to master. In this example I’ve checked out the contact_form_test branch and added a new commit. In order for Tut to help me merge that forward, I need to be able to generate the list of steps.

The correct order here (last step first) is master, custom_form_rendering, contact_form_test, edit_addresses, address_model, confirm_contact_email, contact_detail_view. But you can’t get that out with either date or topo ordering. You really need to walk back from master, looking for branches (head refs), and at each step look for any head refs reachable (as children) that you haven’t already seen. I haven’t figured out how to do that yet with git plumbing commands, so for the time being I’m just using a text file to record the correct order. [1]

I’m really excited about presenting Effective Django at OSCON, and appreciate the feedback and suggestions from everyone.

[1]	I contemplated just using a text file in the repo as the solution, but realized that this has its own issues: if it’s under version control as well, then what’s the “right” version to look at when you’re in a branch? That branch’s? Master’s? It’s not clear to me.

author:	Nathan Yergler
category:	Effective Django
tags:	talks, effective django, python, tut,
comments: