Saturday, December 12, 2009

How I discovered that I was building Situated Software

I'd been creating “situated” software for months before I first stumbled on Clay Shirky's article, which gave a name to what I was already up to. You should stop here and go read the original article, in part because he's a captivating writer, and in part because I've been subsequently marinating in these ideas for more than a year now, and some of what I found earth-shaking on first read I will have forgotten was novel enough to require explanation.

I wrote what follows to explore how situated software changes the design landscape at my day job: writing software inside of Yahoo's IT department. In some sense it's a situated essay; there are assumptions I get to make, working at a company in America, in that odd part called Silicon Valley, with a significant portion of my user base being other software authors – I don't think anybody is going to pick this up and learn much that's immediately and unequivocally applicable to any situation other than mine. But that's okay, even the exercise of thinking about “that wouldn't work here” is still helping you to explore what situated software would look like in your situation.

I would boil “Situated Software” down to “software that doesn't make sense outside of the situation it was designed for.” For contrast, think about Microsoft Word. It's hard to imagine a typed document that wouldn't basically fit in Microsoft Word -- it's as applicable to school papers as it is to filing for bankruptcy or creating Star Trek fan fiction. Microsoft pays a legion of programmers, testers, linguists, and sociologists to make sure Word is as broadly useful as possible; I expect this is why my seventh grade class took a mandatory course about Microsoft Word: whatever our future vocation, Word would be an acceptable (if not ideal) tool.

So if Word isn't situated software, what is? I've been working on two applications that have benefitted from a situated state of mind, and I'll be mining these for examples and anecdotes. One of them is a web front-end for an enterprise phone system, that we call Yahooized Asterisk (Asterisk is the open source phone system that underlies the custom front-end). The other is a web-based project management tool, named Smithers. Part of what's interesting about these examples is that you can buy non-situated software to meet both of these needs, so I'll be explaining where my software is superior, and where (hopefully less frequently) it falls short.

Barrier to Entry
Because both systems are designed to only be used by employees, I can assume that any user will already have an account in Yahoo's single-sign-on solution for employees. Users get a single ID and password that grant access to dozens of situated (and retro-fitted commercial) apps; all these applications already know who the user is without the sign-up speedbump.

Pre-Populated Data
Yahoo, like most large companies, knows volumes about its employees; phone numbers, org charts, geography, mailing list membership, etc. Yahooized Asterisk, for example, looks up all incoming caller ID numbers against the corporate phone directory; if it gets a match, your phone shows the caller's name in addition to his number. (Think of the contacts on your cell phone, but with 20,000 entries, kept rigorously up to date by human resources and the telecom team.)

Smithers uses information from the org chart to figure out which managers to keep in the loop on which projects. For example, if A has a project that is sponsored by a senior vice president D, Smithers knows to keep A's boss (B) and boss's boss (C) in the loop. If C quits and is replaced by C', Smithers learns about it from the HR database, and stays up to date.

A less utilitarian but more personal example in Smithers is the user's profile picture. This is a pretty common feature of consumer software; LinkedIn and StackOverflow and eBay all have it, but most people seem to keep the (ugly, useless) silhouette default. I actually don't recall ever seeing this feature on business software, which is a real shame, because inside a big company seems like one place you could actually get a high attach rate. At Yahoo, on your first day, we take your picture for your security badge, then upload that picture into a web service (which Smithers and other situated apps consume). If you don't like your mugshot, you can replace it with something more flattering, but even the default is meaningfully you.

Humor and In-Jokes

The whole point of situated software is to be more responsive to fewer users than commercial software. But situated software can represent more than a better-tailored feature list, it can participate in a community's culture. Situated software can even have a sense of humor.

For example, this is part of the dialog that greets a user who hasn't set up his voicemail pin in Yahooized Asterisk:
This PIN is all that stands between you and evil monkeys who want to listen to your voicemail and forward your calls. Our crack monkey defense researchers suggest these minimum PIN strength guidelines:

  • the PIN must be all numbers
  • at least 4 digits long
  • not the same as your extension
  • not something a monkey would guess (e.g., 1234 or 9876)

The factual part of the advice is pretty standard for any voicemail system.  Inside the company I can inject a little fun into it, without worrying about what Joel describes as "one old lady in Minnesota" getting bent out of shape about my monkey-incorrectness.

Smithers is named after the loving assistant to the evil Mr. Burns on the Simpsons. Late projects get a tiny icon of the Simpsons' character Krusty the Klown. The user who originally suggested this feature meant it as “your project is making you look like a clown” – a little bit of negative reinforcement to improve behavior. In the jargon of the users, it's now called “Getting a Krusty.”  I'm just grateful that isn't something horrific in Urban Dictionary.

Taking a cue from the Treehouse of Horror, Smithers even had a special Halloween edition. The project status icons changed from boring dots of red, yellow, and green to blood, pumpkins, and frankenstein icons. Even users names were updated in the Simpsons' tradition of “scary” names in the credits.  The change got a lot of buzz, and a lot of goodwill for the app. More importantly, users submitted 40% more updates than the day before.

So if everyone can use these systems, how do they learn that they should? For Smithers, the key was having a patron. When he told the managers that report to him to use Smithers, they passed the edict down to their employees. So Smithers got a big push from a top-down directive, instead of relying on things like excellence to entice users. It's what happened next that I'm more proud of.

Smithers is not the only mechanism Yahoos use to track projects. Some teams use spreadsheets or whiteboards or swarms of sticky notes; others use Microsoft Project, or Serena Mariner, or VersionOne. The only way I could imagine that I could improve on all of these was to tap into the community that needed to use the tool.

One way I did that was to work with several power users: employees who would be generating a lot of input, and mangers who would need structured output. I set up a mailing list to discuss design decisions, drew paper prototypes with users, and even posted the development schedule. Between meeting their needs, and the leavening of humor I mentioned earlier, Smithers became popular with the users, which is great because they didn't actually have a choice in whether or not to use it. But because users liked it, they used it for more projects, and updated it more frequently.

At this writing, Smithers is almost a year and a half old, and 111 users have created 1,934 projects and submitted 13,712 updates.  The original patron has long since left the company, but Smithers is still going strong thanks to the user-generated momentum.

Too Much Flexibility, or Not Enough?
If the great virtue of situated software is that it can become whatever its situation requires, the great tragedy of situated software is that there are always more user needs than developer hours. The whole Yahooized Asterisk system originally grew out of the idea that 95% of Yahoos are using a tiny sliver of the functionality of our commercial phone system. So we decided that if we could replace our expensive commercial phone system with an open source one, and really kick ass on the few “essential” features, we could save a boatload of capital, and actually improve the service. And so far, we've delivered on that vision, for 95% of our users.

But that last 5% is a doozy. First off, that small fraction is actually a big number in absolute terms; 5% of 13,500 full time employees represents 675 not delighted people. And aside from not being happy with my software, they really don't have much else in common. Some are call-center managers, who want better sticks to beat their call-taking zombies with. A few dozen are secretaries and assistants that want to be able to field their bosses' calls, in scattered pockets that look more Madison Avenue than Silicon Valley. One guy wants to use the corporate long distance plan when he's making business calls from his personal cell phone, like a calling card. The M&A team wants a super-secure conference bridge.

This is, of course, not unique to situated software, this is general to design tradeoffs in every field. What is unique is that we don't have cash as a feedback mechanism, and we do have an amplified sense of who the squeaky wheels are. In a commercial software shop, I'd probably rank those features by the number of people who want them, and the price-per-head I thought I could extract for that feature, and complete them in that order. Missing that, I can be guided by politics or the features that are exciting to me.

So where does that leave mass-produced software?
To be clear, I do think that there is still a place -- lots of places -- for mass-produced software. The underpinnings of both Smithers and Yahooized Asterisk are general problems, like databases and web servers and phone systems.  The top level tool is situated, but we're standing on the shoulders of giants like Apache and YUI and Asterisk.

Maybe what's changing is that the unsolved problems in software are getting closer to the promise Hal Abelson lays out in the first lecture of Structure and Interpretation of Computer Programs. If software is actually about formally expressing a process, then it shouldn't be surprising that a company would express something so culture-specific as “how we run projects” in their own image, rather than trying to find something close.

I expect that software will continue to get easier to write, and that this will encourage more organizations to retain staff that can create excellent, custom software in increasingly narrow situations. I think eventually software creation will be mostly an exercise in codifying requirements, and communicating those to sympathetic tools. As we get closer to those tools and that understanding of software, I expect a larger percentage of new software to be situated. 

No comments:

Post a Comment