I read that 2018 was the final year of Open Source Bridge. Reading that I felt sadness, as well as gratitude the organizers were able to choose that. I spoke at the inaugural Open Source Bridge, and my memory of that is that it was such a refreshing vibe compared to the other conferences I was attending at the time (OSCON, Semantic World). There was space for self care (yoga), for weird ideas, and for community in a way I didn’t experience at other conferences. I started to write this as a status update, and realized that these feelings about Open Source Bridge are part of a larger wave of nostalgia for the late aughts I’ve been feeling lately.

The first Open Source Bridge took place in 2009. I had been living in San Francisco for two years and was working at Creative Commons. My role at CC had morphed from “figure out what we could build to engage people with the commons” to “figure out our technical strategy and how we fit with the You Tubes of the world”. I was a lot better at the former; at the very least I enjoyed it more. But there was still something there that I felt energized being part of. There was a community that I appreciated and valued. I’ve reflected in the past that formed the core of this online community for me. It also lived on blogs and in the #cc IRC channels.

So I guess it’s appropriate that some of this nostalgia is undoubtedly triggered by all the awesome Indie* work being done. Just this week I learned about Indie Book Club and Indie Web Ring. And while both are simple, that’s sort of the point: I’m happy they both exist, because [I hypothesize] they help me connect with a larger community while being my whole self online. Being my whole self means that I “show up” in a solid, singular way: you come to my blog and get printmaking, scifi quotes, Python advice, sewing; you get me.

My talk at the inaugural Open Source Bridge was entitled “A Database Called the Web”. It was the second and last time I presented that talk, which was a shame because I don’t think I ever really got the kinks worked out. Creative Commons was founded with this technical layer under girding the licenses, and “A Database Called the Web” was my attempt to articulate that decentralized, federated vision in a way that didn’t start with RDF, XHTML, etc. And that’s why in addition to feeling some nostalgia I also feel some hope: it seems like with ~ 10 years of time (and a lot of heartache) people have moved on to building decentralized things that they want to see exist in the world. And that makes me happy.

“Effective Django” at OSCON

I’m going to be presenting my introductory Django tutorial, Effective Django at OSCON later this month. If you’re going to be at OSCON and haven’t selected your tutorials yet, or just think a trip to Portland, Oregon sounds nifty, there’s still time to sign up. You can find the details on the OSCON tutorial page.

In preparation for that I’ve been continuing to work on the content. I presented the tutorial to some of the Eventbrite engineering team a couple weeks ago, and thier feedback was very useful. In response, I’ve made a few changes. Specifically, I split up the Views chapter with a brief interlude on static assets and template inheritance. It’s something that I didn’t cover the first time around, but based on the questions, I think some guidance is useful.

The revisions for OSCON also include updating the sample code repository that goes with the tutorial. I developed a tool, Tut, to help manage these stacked branches, and while making changes to early parts of the tutorial code, I realized it still requires significant work to really be a good workflow tool. One of the most important requirements for Tut is the ability to manage an ordered series of “checkpoints” and move between them.

When I started working on the sample code this time around, I was on a new laptop, so I had to start from a fresh clone. This was revealing and frustrating. I discovered Tut assumed all the branches were already local, which they obviously aren’t with a fresh clone. Worse, the git magic I was trying to use to get the branch list in the “right” order was pretty fragile, and broke when I tried to lean on it at all.

This screenshot shows a common editing case. My intention is to manage each step, or checkpoint, in the tutorial as a branch. Each step builds on the previous one, so if I make a change to something early in the tutorial, I just need to merge the branches “forward” until I get to master. In this example I’ve checked out the contact_form_test branch and added a new commit. In order for Tut to help me merge that forward, I need to be able to generate the list of steps.

The correct order here (last step first) is master, custom_form_rendering, contact_form_test, edit_addresses, address_model, confirm_contact_email, contact_detail_view. But you can’t get that out with either date or topo ordering. You really need to walk back from master, looking for branches (head refs), and at each step look for any head refs reachable (as children) that you haven’t already seen. I haven’t figured out how to do that yet with git plumbing commands, so for the time being I’m just using a text file to record the correct order. [1]

I’m really excited about presenting Effective Django at OSCON, and appreciate the feedback and suggestions from everyone.

[1]I contemplated just using a text file in the repo as the solution, but realized that this has its own issues: if it’s under version control as well, then what’s the “right” version to look at when you’re in a branch? That branch’s? Master’s? It’s not clear to me.
author:Nathan Yergler
category:Effective Django
tags:talks, effective django, python, tut,

“Effective Django” at PyCon 2013

PyCon (the US variation, at least), is about a month and half away, and once again I’m looking at the schedule of presentations and events and wondering how it is the community pulls it off every year. I’m also busy preparing my contribution to PyCon. This year I’m happy to be presenting a tutorial, Effective Django.

You may wonder what I mean by “Effective Django”. It’s an introduction to Django with a focus on good engineering practices. What I’ve noticed from my own experience over the years is that with all of its features and flexibility, Django makes it easy to get up and going really quickly. It also lets you write code that’s difficult to test, scale, and maintain. I have written plenty of code like that over the years, and the problem is that the real pain may come long after the initial implementation. From talking to engineers at Eventbrite and elsewhere I have learned that I’m not alone in this, so I’ve been working on documenting how to do leverage Django effectively. My goal is that attendees of the tutorial will leave feeling like they’re able to work on a Django application and identify things to do (and avoid) that will help them write code that’s cohesive, testable, and scalable.

I’m enjoying putting together the material for PyCon, and I hope that if you’re new to Django and interested in starting off on the right foot you’ll join me in Santa Clara for the afternoon.

If you’re totally new to Django and want to get a complete introduction to web apps and Django, Effective Django pairs well with Karen Rustad’s “Going from web pages to web apps“.

author:Nathan Yergler
tags:talks, effective django, python,

CiviCon Plenary: What Are We Paying For?

I’m spending part of my day today at the first ever CiviCon, a one day conference for the CiviCRM community. I was honored (and a little surprised) to be asked to give the opening plenary talk. My original travel schedule called for me to fly back from Istanbul yesterday, get up this morning and present (and then presumably crash from jet lag). Luckily (for me) the volcano derailed my trip to Istanbul, so I was able to present with a little more sleep. This is the text of the talk I prepared.

Good morning. I am incredibly happy to be here talking to you this morning. I’m excited to be here because CiviCRM has been and continues to be a very important part of our infrastructure at Creative Commons. Beyond CC, I think it’s an important piece of software for non-profit and grassroots organizations, one that should not be ignored when evaluating donor and constituent management applications. This morning I want to tell you about my experience with it, and why I think it’s so very, very important.

I’ve been at Creative Commons about six years now. I started working as an engineer, which basically meant I wrote code, deployed code, and babysat the server — there was only one then — when it got cranky. We started using Civi in 2006. Creative Commons is a 501c3, which means we have to work towards passing the IRS’s public support test: 30% of our operating budget from “the public”, in our case. In 2005 we had a rather unpleasant realization: we hadn’t done shit, and we were getting close to the end of our five year grace period. I hacked together a contribution page, wired up PayPal for donation processing, and put together a terrible looking Plone backend that allowed people in the office to enter offline checks, promised contributions, etc, and to make the all important thermometer work on the front page. And we started begging. Creative Commons is fortunate to have a generous, passionate community, and they responded to our call. After the dust had settled and we realized we were going to have to do this again, we knew we needed something better than what I could write in between other projects. It was clear that we didn’t have the resources to build something complete, and still fulfill our mission, so we started looking around. Mike Linksvayer, then our CTO, asked me if I’d used Raiser’s Edge before, and if so, what I thought of it. Raiser’s Edge had — perhaps has — the mindshare, and our development director at the time was certain Raiser’s Edge was the essential piece of technology to make the jump to the big leagues. If I recall correctly, my response is not suitable for mixed company.

Before working at CC, I worked for a very different kind of non-profit. Canterbury School is a private, K-12, college prep school. With small classes, dedicated teachers, and a commitment to education that includes computer programming for every student, they are they best game in private education in northeast Indiana. The faculty and staff there do an amazing job. Every year Canterbury holds an annual campaign, which raises funds to support the school operating budget. Part of my job there was supporting the development office and Raiser’s Edge. Our Raiser Edge installation had one primary user, Barbara. I am convinced Barbara is the direct descendant of the bearded UNIX sysadmin wizards. Unlike her ancestors, she does not keep mainframes online, but rather has her own special wizard-like skills: comprehensive knowledge of the Raiser’s Edge query language and interface. So you can imagine that days when Barbara called my office were not my favorite days at work. “The database is locked” “My labels template won’t print right” “The query isn’t returning data I know is in there.”

Now in all fairness, this was almost ten years ago, and we were running Raiser’s Edge on Sybase SQLAnywhere. But I’d also like to point out that we were paying about $3000 a year for a support contract, a contract sold with a statement along the lines of, “You want updates when they come out, don’t you?” These updates and improvements, when they did come out, were usually accompanied by new system requirements, indirect costs, and lots of time spent making sure the upgrade went correctly. And bugs fixes? What bugs?

This, my friends, is what we call a protection racket.

I also had enough experience to know that expecting to deploy a CRM package out of the box with no customization was probably unreasonable. I had worked on a CRM and ERP deployment in 2003 for an international manufacturer that took the better part of a year, and whose customization costs were almost as high as the software licensing costs. I was lucky on that project: I got to write C++ DLLs to handle custom accounting information from pricing tables. And this was a product sold on how open and customizable it was. “Look, we have an API, you can write a DLL.” Gee, thanks.

So back to Creative Commons: after some consideration, CiviCRM was selected as a platform. When we started out with Civi, we paid for CivicActions’ development, customization, and hosting services. We also paid for updates, improvements, and upgrades, just like we would have with Raiser’s Edge. It’s a couple years later, in 2007, when the difference between the two really becomes apparent. because

In 2007, we had been been using Civi for a couple of years, and while we weren’t unhappy with it, it still wasn’t really a piece of software any of us looked forward to using. I personally tried to ignore it as much as possible except during the lead up to the annual campaign. It worked, it did it’s job, but it felt quirky and a little clunky, sort of like I was working against it sometimes. But there was hope. The 3.0 release was going to be a big upgrade. Lots of new features, improved user interface, and lots of attention paid to usability. We talked to CiviActions about a 3.0 upgrade and some additional customization we wanted to do. The quotes we got back were, well, expensive. Now I don’t think they were out of line, but it was more than we had budgeted at the time. Our situation internally had changed, as well. We were up to a handful of servers, two full time engineers, and a graphic designer with great technical chops. Instead of quoting out every change and upgrade, it started to feel like we might be ready to walk on our own.

So we checked out the source code — including all the custom code CivicActions built for us — and installed it on our own server.

I think this bears repeating. We downloaded the source code — the same source code our vendor was using — and installed it with our own tools. We didn’t have to ask for permission, pay a fee, or “upgrade” our license; it was already available to us.

You see, unlike the protection racket I lived under at Canterbury, at Creative Commons we’ve been paying for something that we own. That the community owns. We’re paying for value that we retain, that we can take elsewhere when we’re ready. This is huge.

Now, for those of you who are open source and free software developers, those of us who run Linux on our laptops, who build applications using open source tools, this sounds like business as usual. But CiviCRM is very special software: it is not software for Geeks, it’s software for Humans.

At Creative Commons, we build legal tools that help people share their creative works with the permissions that they choose, the some rights reserved that work best for them. But we also recognize that most of the time, those creators are not lawyers. They’re coming to us because they’re not lawyers, and our job is to reduce the number of hoops they have jump through to share their work. So we make our tools available for two different audiences, which we jokingly refer to as Lawyers and Humans. Today I’d like to posit that we have a similar divide when it comes to free software. Apache, Linux, MySQL, CouchDB — these are all examples of software for Geeks. CiviCRM is software for Humans. And it’s really important software for Humans.

A friend of mine, Asheeesh Laroia, runs Open Hatch, a website dedicated to helping people get involved with open source software. Talking about software like CiviCRM, he made the comment, “It’s important that we give communities tools that they can use and that they can control. Otherwise how do we expect them to be independent and self sustaining?” Fundraising and constituent management is a critical part of any non-profit’s life cycle. Why do we think it’s OK for these organizations — many of which we personally contribute to — to enrich the coffers of a for profit businesses with no real long term return?

Please don’t misunderstand me: I am not saying that paying money for support is wrong, misguided, or unncessary. I am saying that we — non-profits — can do a lot better than investing in a protection racket. We can pay for support, and at the same time invest in our tools. And when we’re ready to move to the next level, there is no upgrade fee. We already own it all.

Creative Commons’ story did not end by simply installing CiviCRM onto our server and moving on. And putting it on our own server has not resulted in the cost of Civi going to zero. In fact, we still “pay” for support, but we do it differently these days. Maintenance and support for Civi is a core responsibility for the technical team today. When our development team has questions, wants to run new reports, or wants to do something new like personal campaigns, they sit down with Chris, John, and Nathan, and figure out what we’ll need to do to Civi to make it work.

Last year we decided that the contribution workflow was too long. We were demanding information from our donors that we didn’t necessarily need, and duplicating information they might give PayPal when processing their payment. We wanted to streamline the process, reducing the number of clicks between us and a donor’s money. We invested the time to implement some custom code on Civi to make this happen. Right now we want to offer tighter integration between CiviCRM and CC Network accounts, a premium we offer donors. So we’re writing code to do that.

The click-streamline code is available to anyone who wants to use it. We’re writing our integration code with the hope we can contribute it to the CiviCRM core. It’s tempting to say that we’re no longer customers, that we’ve moved up to become partners. But the truth is that we were never just customers: with CiviCRM, we were partners from day one, we just didn’t necessarily realize it.

Now I know that lots of commercial vendors talk about partnering with customers. I’d like to call bullshit on that. If you don’t have the source code, you are not a partner. If you can’t be trusted to inspect the software you rely on, you are not a respected equal in that relationship. Companies go out of business. Companies are acquired. Management changes. This is the reality of business.

Let’s go back and look at Canterbury again. I sent Vern, Canterbury’s Director of Technology and a man who can really only be called my mentor, an email to fact check my memory about what we were working with in 2001. Vern told me he’s part of a Google Group formed for schools to exchange SQL Server queries for accessing their data. Why SQL Server queries and not an API — seems a little crude to me. Turns out, if you want to write your own code to access your data through their API, there’s a toll: $10000 and another $1000 per year in support.

If a customer want to access their data in the safest, sanest manner, they have to pay. Again. If that were the reality for CiviCRM, we would not have been able to streamline our contribution process. We would not be integrating CC Network memberships as a premium for donating. Hell, I’m not even sure we’d be able to show the beloved thermometer on the front page. The cost is too high.

Let me wrap up with some thoughts about the future. If you work for a non-profit or grassroots organization, I believe it is incumbent upon you to think about things in terms of fiscal stewardship. Where is your money going today, and what value are you getting? Are you a customer, or are you a partner? As an officer at a non-profit, I see fundraising and development as critical to our ability to execute our core mission. Our mission at Creative Commons is not to make money, but if we can’t keep the lights on, we can’t help people share their work online.

If you’re using CiviCRM today, it’s time to start talking about it. I find that lots of non-profits I talk to are unaware that there’s a different way. They haven’t thought about the fact that they have a choice. We need to work to increase the mind share that CiviCRM has. If you think Civi is only good for small organizations, that you can’t really recommend it for large scale installations, I’d like to challenge that misconception. Mozilla Foundation. Wikimedia Foundation. Creative Commons. We’re all using CiviCRM for all or part of our constituent management solution. And if you’re a user or integrator who’s customizing it, it’s time to start thinking about how you give back. The AGPL already requires that you make your changes available. I think it’s worth your while to consider how you’re making those changes. Is this a customization someone else might want? If so, I encourage you to take a few minutes, talk to the developers on IRC or the forums, and think about how you might build your solution in a general way. The CiviCRM developers are some of the most available and responsive I’ve ever encountered. They can give you guidance about how to think about a problem. It may take some extra time to build a general solution. But if you can contribute that back to the core codebase, you’re going to make upgrades much easier. And the great thing is that at the same time you’re making things easy for yourself in the future, you’re also helping to sustain the community and the development process.

Finally, if you’re evaluating CiviCRM today, I have some advice for you. Do not think to yourself, “oh, I can download this for free, this isn’t going to cost me anything.” Do not look at CiviCRM as a zero cost solution. My experience is that there is no such thing in CRM, and I’d argue you wouldn’t want there to be: we all need experts we can call upon in an emergency, and paying for a tool enables the developers to improve it. Instead, do your checkbox evaluation and realize that Civi is competitive with commercial packages. Talk to people who are using CiviCRM and tell them how you plan to use it, ask them about their experience. Then ask yourself, “how much am I going to spend year over year, and what am I getting from that? How is my investment going to grow? Am I paying for protection, or am I investing in my organizations future stability?” I think the answer is self evident.

Working at Creative Commons has been an amazing opportunity for me, and I’m proud of a lot that I’ve accomplished over the past six years. But I’m especially proud of how we’ve used and contributed to CiviCRM. I’m proud to be a user, and I feel good every time I talk to my team about Civi and hear, “Oh, yeah, I just jumped on IRC and got an answer from one of the core developers.” You can not buy that sort of dedication and support.

date:2010-04-22 11:41:29
tags:cc, civicon, civicrm, talks