Sunday, December 13, 2009


Where did all these words come from?
At Yahoo!, there is an internal mailing list called 'devel-random.'  This list receives hundreds of messages a week from Yahoo! employees, on topics ranging from technology to company politics to global politics, and is read by an equally diverse population, including developers, project managers, marketers, and both founders.

So when I took a Linguistics class from Stanford's CSP program, I decided to use a copy of this list's archives I had laying around: 8,000+ messages sent between January 31st and May 28th, 2008.  I sketched up a script in PHP that downloaded these emails out of my Exchange account, split them up into words, then threw out all the words that matched the GNU Aspell dictionary.  The rest of the words I reviewed using an AJAXy front end, throwing out obvious misspellings, proper nouns, and bits of URLs, but providing me with enough context from the original message to spot the diamonds in the rough.  What's left were interesting enough to get a class paper out of.

If you are a developer or an IT nerd (like me!) a few of these are going to seem old hat to you.  Several of them were already in off-the-beaten-path dictionaries like the Hacker's Dictionary or the Urban Dictionary at the time of my project.  But hopefully you'll find something new to you, or bent into a new use, or maybe just a better definition here that helps you communicate with your "normal" friends.

So, what was interesting?
I’ll break down the interesting words into three categories.  The first is jargon, words that add detail or speed up conversation in our areas of expertise.  The second are terms that I think have the potential to spread beyond technologists and into mainstream usage.  The third are words used when talking about the generic use of lower-case "google" to mean "searching the internet" without necessarily referring to the Google brand (which, as you might imagine, really pisses Yahoos off).

Dogfood (noun or verb) – This term has its origin in the expression “to eat our own dog food.”  In technology, this means “using our own products.” Dogfooding is, for programmers, a joyous milestone, because it means the product is usable -- at least by someone predisposed to see past the current bugs.  The term is also open to parody, since the expression literally means people are consuming something not fit for human consumption.

Mashup (noun), mashable (adjective) – Both these words came into my experience through the hack culture at Yahoo.  A mashup is an application that uses data or functionality from two existing applications, usually applications that you did not write and whose development you do not directly influence.  Douglas Crockford, a programming language architect at Yahoo, says “Mashups are the most interesting innovation in software development in decades.”   Mashable is a complement, meaning “lends itself to mashups.”

Spidered (verb) – Meaning “collected by a spider,” this usually comes up in context of things that search engines are capable of finding, as opposed to content that is deliberately or accidentally not visible to search engines.  Spider is a metaphorical term for the web crawlers that find content for inclusion in search engines.

Threadsafe (adjective) – A threadsafe program is one that is demonstrably error-free when several copies run simultaneously and share some resource, like memory. It is a fairly recent development for consumer-grade hardware to support this possibility, but it's not widely implemented in software.  Rasmus Lerdorf, creator of the PHP programming language has said "Most humans are simply not smart enough to write threadsafe code.

Migrated (verb, passive voice) – The interesting nuance that I credit to the technology industry is the passive voice usage.  In general usage, the person or animal that migrates is the actor; in technology a user can be migrated, without their involvement or consent, between technologies or products.

AJAXy (adjective) – This term stems from the programming trick “Asynchronous JavaScript and XML” that powers modern web tools like GMail and Flickr.  The adjective form can either mean “literally uses AJAX” or more figuratively “showy, especially but not necessarily by using AJAX.”

Freetard (noun) – Refers to someone who conflates free as in open source and free as in zero cost; a freetard undervalues other people’s work, and demands that everything delivered digitally cost zero dollars, regardless of whether the creator wishes to participate in open licensing or not.

GA'ed (adjective) – A "GA release" is the milestone that makes a product "Generally Available."  This form, pronounced “gee-ayd” effectively means “made generally available.” 

Folksonomy (noun) – A taxonomy built by users.  Usually this implies that a rigid hierarchy will not be imposed, and that one item may live in many categories.

Mainstream Potential
Betamaxes (noun) – Used as a general term for any dead end technology, especially technologies that had strong vendor backing, but were killed either by user preference or published content.  The term primarily applies in winner-take-all fields powered by network effects: as betamax slipped in popularity, the tapes weren't stocked in stores, which made consumers not want the players, which caused a negative feedback loop.

Frankensteined (verb) – To put something together out of existing parts, but with a strong pejorative tone.  Frankensteined has the added connotations of shoddy workmanship, poor performance, and unmaintainability.  (Contrast with mashup.)

Offlist (adverb) – Compounding of “off of the list.”  E.g., “I will reply offlist.”  I suspect it owes it’s shape by comparison to “off-the-record,” and it tends to be used in either the “I don’t want to bother the list with this” or the “I would rather keep this private from the list” sense.

Spim (noun) – Spam delivered by instant messenger instead of email.  This appears to be one of a series of refinements on the general term spam, which also includes spit (Spam over Internet Telephony) and bacn

Spamfighting (gerund) - The act of fighting spam, by comparison with firefighting.  I like that his gives a heroic nuance to the forces arrayed against spam.

Verbing Weirds Language
Verbiness (noun) – A measure of how well a noun lends itself to unambiguous verb conversion.  Yahoo is considered to have low verbiness, Google has high verbiness.  The jury is still out on the verbiness of "Bing."

Kleenexification (noun) – The transition through which a brand name comes to stand in for a generic activity or product.  Like betamaxes above, using this example to stand in for a whole class of activity evokes the whole lore of the specific example.

Genericize (verb) – To make something generic.  This also implies “to erode the claim to trademark.”

Saturday, December 12, 2009

How I discovered that I was building Situated Software

I'd been creating “situated” software for months before I first stumbled on Clay Shirky's article, which gave a name to what I was already up to. You should stop here and go read the original article, in part because he's a captivating writer, and in part because I've been subsequently marinating in these ideas for more than a year now, and some of what I found earth-shaking on first read I will have forgotten was novel enough to require explanation.

I wrote what follows to explore how situated software changes the design landscape at my day job: writing software inside of Yahoo's IT department. In some sense it's a situated essay; there are assumptions I get to make, working at a company in America, in that odd part called Silicon Valley, with a significant portion of my user base being other software authors – I don't think anybody is going to pick this up and learn much that's immediately and unequivocally applicable to any situation other than mine. But that's okay, even the exercise of thinking about “that wouldn't work here” is still helping you to explore what situated software would look like in your situation.

I would boil “Situated Software” down to “software that doesn't make sense outside of the situation it was designed for.” For contrast, think about Microsoft Word. It's hard to imagine a typed document that wouldn't basically fit in Microsoft Word -- it's as applicable to school papers as it is to filing for bankruptcy or creating Star Trek fan fiction. Microsoft pays a legion of programmers, testers, linguists, and sociologists to make sure Word is as broadly useful as possible; I expect this is why my seventh grade class took a mandatory course about Microsoft Word: whatever our future vocation, Word would be an acceptable (if not ideal) tool.

So if Word isn't situated software, what is? I've been working on two applications that have benefitted from a situated state of mind, and I'll be mining these for examples and anecdotes. One of them is a web front-end for an enterprise phone system, that we call Yahooized Asterisk (Asterisk is the open source phone system that underlies the custom front-end). The other is a web-based project management tool, named Smithers. Part of what's interesting about these examples is that you can buy non-situated software to meet both of these needs, so I'll be explaining where my software is superior, and where (hopefully less frequently) it falls short.

Barrier to Entry
Because both systems are designed to only be used by employees, I can assume that any user will already have an account in Yahoo's single-sign-on solution for employees. Users get a single ID and password that grant access to dozens of situated (and retro-fitted commercial) apps; all these applications already know who the user is without the sign-up speedbump.

Pre-Populated Data
Yahoo, like most large companies, knows volumes about its employees; phone numbers, org charts, geography, mailing list membership, etc. Yahooized Asterisk, for example, looks up all incoming caller ID numbers against the corporate phone directory; if it gets a match, your phone shows the caller's name in addition to his number. (Think of the contacts on your cell phone, but with 20,000 entries, kept rigorously up to date by human resources and the telecom team.)

Smithers uses information from the org chart to figure out which managers to keep in the loop on which projects. For example, if A has a project that is sponsored by a senior vice president D, Smithers knows to keep A's boss (B) and boss's boss (C) in the loop. If C quits and is replaced by C', Smithers learns about it from the HR database, and stays up to date.

A less utilitarian but more personal example in Smithers is the user's profile picture. This is a pretty common feature of consumer software; LinkedIn and StackOverflow and eBay all have it, but most people seem to keep the (ugly, useless) silhouette default. I actually don't recall ever seeing this feature on business software, which is a real shame, because inside a big company seems like one place you could actually get a high attach rate. At Yahoo, on your first day, we take your picture for your security badge, then upload that picture into a web service (which Smithers and other situated apps consume). If you don't like your mugshot, you can replace it with something more flattering, but even the default is meaningfully you.

Humor and In-Jokes

The whole point of situated software is to be more responsive to fewer users than commercial software. But situated software can represent more than a better-tailored feature list, it can participate in a community's culture. Situated software can even have a sense of humor.

For example, this is part of the dialog that greets a user who hasn't set up his voicemail pin in Yahooized Asterisk:
This PIN is all that stands between you and evil monkeys who want to listen to your voicemail and forward your calls. Our crack monkey defense researchers suggest these minimum PIN strength guidelines:

  • the PIN must be all numbers
  • at least 4 digits long
  • not the same as your extension
  • not something a monkey would guess (e.g., 1234 or 9876)

The factual part of the advice is pretty standard for any voicemail system.  Inside the company I can inject a little fun into it, without worrying about what Joel describes as "one old lady in Minnesota" getting bent out of shape about my monkey-incorrectness.

Smithers is named after the loving assistant to the evil Mr. Burns on the Simpsons. Late projects get a tiny icon of the Simpsons' character Krusty the Klown. The user who originally suggested this feature meant it as “your project is making you look like a clown” – a little bit of negative reinforcement to improve behavior. In the jargon of the users, it's now called “Getting a Krusty.”  I'm just grateful that isn't something horrific in Urban Dictionary.

Taking a cue from the Treehouse of Horror, Smithers even had a special Halloween edition. The project status icons changed from boring dots of red, yellow, and green to blood, pumpkins, and frankenstein icons. Even users names were updated in the Simpsons' tradition of “scary” names in the credits.  The change got a lot of buzz, and a lot of goodwill for the app. More importantly, users submitted 40% more updates than the day before.

So if everyone can use these systems, how do they learn that they should? For Smithers, the key was having a patron. When he told the managers that report to him to use Smithers, they passed the edict down to their employees. So Smithers got a big push from a top-down directive, instead of relying on things like excellence to entice users. It's what happened next that I'm more proud of.

Smithers is not the only mechanism Yahoos use to track projects. Some teams use spreadsheets or whiteboards or swarms of sticky notes; others use Microsoft Project, or Serena Mariner, or VersionOne. The only way I could imagine that I could improve on all of these was to tap into the community that needed to use the tool.

One way I did that was to work with several power users: employees who would be generating a lot of input, and mangers who would need structured output. I set up a mailing list to discuss design decisions, drew paper prototypes with users, and even posted the development schedule. Between meeting their needs, and the leavening of humor I mentioned earlier, Smithers became popular with the users, which is great because they didn't actually have a choice in whether or not to use it. But because users liked it, they used it for more projects, and updated it more frequently.

At this writing, Smithers is almost a year and a half old, and 111 users have created 1,934 projects and submitted 13,712 updates.  The original patron has long since left the company, but Smithers is still going strong thanks to the user-generated momentum.

Too Much Flexibility, or Not Enough?
If the great virtue of situated software is that it can become whatever its situation requires, the great tragedy of situated software is that there are always more user needs than developer hours. The whole Yahooized Asterisk system originally grew out of the idea that 95% of Yahoos are using a tiny sliver of the functionality of our commercial phone system. So we decided that if we could replace our expensive commercial phone system with an open source one, and really kick ass on the few “essential” features, we could save a boatload of capital, and actually improve the service. And so far, we've delivered on that vision, for 95% of our users.

But that last 5% is a doozy. First off, that small fraction is actually a big number in absolute terms; 5% of 13,500 full time employees represents 675 not delighted people. And aside from not being happy with my software, they really don't have much else in common. Some are call-center managers, who want better sticks to beat their call-taking zombies with. A few dozen are secretaries and assistants that want to be able to field their bosses' calls, in scattered pockets that look more Madison Avenue than Silicon Valley. One guy wants to use the corporate long distance plan when he's making business calls from his personal cell phone, like a calling card. The M&A team wants a super-secure conference bridge.

This is, of course, not unique to situated software, this is general to design tradeoffs in every field. What is unique is that we don't have cash as a feedback mechanism, and we do have an amplified sense of who the squeaky wheels are. In a commercial software shop, I'd probably rank those features by the number of people who want them, and the price-per-head I thought I could extract for that feature, and complete them in that order. Missing that, I can be guided by politics or the features that are exciting to me.

So where does that leave mass-produced software?
To be clear, I do think that there is still a place -- lots of places -- for mass-produced software. The underpinnings of both Smithers and Yahooized Asterisk are general problems, like databases and web servers and phone systems.  The top level tool is situated, but we're standing on the shoulders of giants like Apache and YUI and Asterisk.

Maybe what's changing is that the unsolved problems in software are getting closer to the promise Hal Abelson lays out in the first lecture of Structure and Interpretation of Computer Programs. If software is actually about formally expressing a process, then it shouldn't be surprising that a company would express something so culture-specific as “how we run projects” in their own image, rather than trying to find something close.

I expect that software will continue to get easier to write, and that this will encourage more organizations to retain staff that can create excellent, custom software in increasingly narrow situations. I think eventually software creation will be mostly an exercise in codifying requirements, and communicating those to sympathetic tools. As we get closer to those tools and that understanding of software, I expect a larger percentage of new software to be situated.