I recently stumbled upon references to the Joel test in job postings on the Stack Overflow Careers 2.0 website (careers.stackoverflow.com). Joel Spolsky (one of the founders of Stack Exchange, the company that publishes Stack Overflow and Careers 2.0) published his “Joel Test: 12 Steps to better code” in a blog entry over 11 years ago – in the fall of 2000.
Some of the steps in his test are still relevant, but not as much of an issue today. Some are antiquated. These are yes/no questions. He suggests that if a team has a score of 10 or lower, the team has serious problems. Here’s my take on his 12 steps, particularly in light of the popularity of more collaborative and agile software development methodologies, like XP and Scrum.
- Do you use source control?
- Still relevant, but I’d say if you don’t have source control, you have serious problems, regardless of the rest of the questions. There’s simply no excuse today with all of the open source options.
- Still relevant, and many folks have caught up with this capability with the tools available today compared to 11 years ago. In fact, the problem has moved beyond dev builds (which I presume he is discussing here) to deployment to different environments (QA, UAT, etc.). Deployment in one step is rare, but desired. This should be the new test, I think.
- This should say “at least” daily. Continuous Integration is the current practice for evolved teams.
- I don’t see this as relevant. On agile teams, for instance, the team is collectively responsible for managing quality. A card on the card wall representing the bug can be a great mechanism for managing your bugs. I suspect this made the cut because Joel’s company’s flagship product at the time was fogbugz – a bug tracking tool.
- I don’t like this one simply because the priority of what gets done is based on business needs. If the bug you want to fix first is a typo in the copyright statement, sure it should be fixed, but not necessarily right away. It should be prioritized against everything else.
- Agile projects emphasize continuous planning. I would rephrase this to ask if planning is done on a regular basis and if everyone on the team understands where the project stands and what needs to be done to get it to the next release.
- I’m not quite sure what kind of spec he’s talking about. Spec is very generic, and could refer to a functional spec (i.e. requirements), architecture spec, design spec, security spec, etc. In an agile environment, we emphasize evolving the requirements and design over the course of the project, rather than stating it in a document up front. I might reframe this to say “Are requirements and design decisions managed and properly understood by the whole team?”
- Many agile teams prefer an open collaborative environment over quiet corners for programmers to crank out code on their own. Pairing is a great model for programming collaboration and certainly makes for a less-than-quiet environment.
- I think the reference here to money is antiquated. Back in the 90’s, open source was not nearly as prevalent. I think I would say “Do you use the best tools for the job?” or something like that. Then again, ask an IntelliJ guy to debate Eclipse, or Maven vs. Ant, or … you get the picture. “Best” is interpreted through the eyes of the technology religion sometimes.
- I would ask “Are testing responsibilities understood and covered by members of your cross functional team?” for an agile environment. “Having” testers does not imply much – particularly if they test the code at the 11th hour – right before release.
- I like this very much. But I wouldn’t say it’s necessarily “during” the interview, perhaps simply “part of the interview process” (e.g. code submissions in advance). Still – sitting down with someone to write code is a great test of a programmer’s skill.
- I would ask instead: “Is the customer represented within your process to provide regular feedback on your product?” Hallways are interesting, but at a software firm, you get a “software developer” slant on the feedback. This is probably good if you’re developing Fogbugz or an IDE, but if you’re creating a website to appeal to the retired community, your usability testing will likely be misguided.
I’ve seen companies in Careers 2.0 convey their Joel Test as “8 out of 12” and “10 out of 12”. Joel’s perspective is that anything below an 11 suggests you have serious problems. If you want to use his test as a metric, OK, but Joel’s view (11 years ago, anyway), is that it represents serious problems. I’m not sure you want to convey that to your potential candidate pool. Or… maybe you do. It certainly is more candid and honest than not pointing out the issues.
Aside: I wonder if Joel would still tout Microsoft’s preeminence as he does in the article (“companies like Microsoft run at 12 full time”). I wonder what Joel would say.