Code Review Tools

The problem with blogging is that you have to actually write the blog posts. Instead of blogging, I’ve been spending a lot of my time dealing with an ongoing issue at my current employers — making code reviews easier to do. Of course, it would help immensely if there was actually a good tool to use. Here’s a business opportunity for anyone looking for one!

What Kate wants in a code review system:

1. Web interface
2. Allows post-commit code reviews
3. Allows pre-commit code reviews (via patch submission)
4. Is fast
5. Understands git
6. Can handle a large number of different repositories
7. Allows both inline comments and block comments
8. Allows auto-creation of reviews
9. Has a straightforward workflow
10. Did I mention, is fast?

Nice to have:

1. Understand other popular version control systems (svn, CVS, Perforce, Mercurial)
2. Understand language syntax
3. Allow some flexibility in workflow

I think I’ve evaluated nearly every code review tool available. My company settled, awhile ago, on Atlassian’s Crucible (with FishEye), but we’ve found it to be unsatisfactory in a number of ways. The basic code review workflow is awesome, and the UI is pretty good (somehwat annoyingly, it used to actually be better, but Atlassian “improved” it). Unfortunately we’ve had an endless series of performance problems with it and it still doesn’t seem to deal with git very well. Plus it seems to have a “one step forward, two steps back” kind of release cycle — every major upgrade solves some problems but introduces a new set of headaches. As a result, for our most complex git repository we set up a ReviewBoard instance because FishEye+Crucible simply couldn’t deal with it. It works fine for our other repositories but the performance is still painful (and yes, it is running — by itself — on some pretty hefty hardware).

ReviewBoard is open source. It’s not bad, but it’s based on a pre-commit code review workflow, which isn’t how my team currently works. You can do post-commit reviews, but either way you need to provide a diff for it to work with; it doesn’t index the code or anything for you like FishEye does. On the other hand, it’s very lightweight, and we have a script we wrote that automatically creates reviews for each checkin. The UI is not as spiffy as Crucible’s, either — but it has the significant advantage at the moment that it actually works with the complex repo we’re unable to get working with FishEye+Crucible.

So, recently I’ve been back in the market for a code review tool that can handle everything we do. I’ve been looking at Gerrit, which uses git, and is based on Guido van Rossum’s Rietveld, itself an open-source, Subversion-based version of Google’s internal, Perforce-based code review tool, Mondrian. Gerrit is used by the Android project, along with a git wrapper called Repo. (What is it with Google and wrappers, by the way?) But Gerrit is a bit … quirky. For instance, instead of just pushing your code, you push it to a special reference, which requires that you use some magic syntax:

git push ssh://sshusername@hostname:29418/projectname HEAD:refs/for/branchname

I haven’t dug into the code, but I presume that this is to also allow you to bypass code review by pushing directly to the repository. (They’re using some hooks and so on to stash your push until it completes review.) Gerrit also wants to manage your Git repositories, which is okay, but we already use gitosis. Still, of the various options I currently have, it looks the best.

In the past, I looked at SmartBear Software’s Code Collaborator, which is similar to Crucible, but rejected it because their git support was almost non-existent. I will, however, be taking a second look, as they claim now to have improved that. On paper, at least, it looks like an excellent candidate; cost might be a serious consideration, however. (It’s rather more expensive than Crucible.)

Does anyone know of any other tools, open source or otherwise, that I should consider? What do you look for in a code review tool, if you use one?