Djblets REST API improvements

We maintain a Django application/utility framework called Djblets. Amongst other things, this provides the framework for implementing REST APIs. We use this for the Review Board API.

Review Board adds a utility layer on top of this that supports a bunch of other features, including API tokens, base test cases and utilities for API unit testing, API resource module registration/lazy lookups, and more. These are all pretty useful things, so we’ve decided to move them in to Djblets.

What this means is that, starting in Djblets 0.9, any consumers of the API will be able to offer:

  • API tokens, with access control defined by a powerful JSON policy definition language.
  • An easier, more complete way to test APIs.
  • Helpers to make more complex API codebases manageable.
  • Utilities for implementing flexible API queries (less than, greater than, etc. for numeric fields, for instance).

This also lets us slim down the Review Board API codebase just a bit, and keep our API support consistent on future projects.

Read More

Some diff viewer polish going into 2.5

In Review Board 2.5, we’re working on polishing up bits of the UI, while at the same time keeping a level of familiarity. We’re not going for a radically changed UI, but rather something that feels more like an evolution of what we had.

So tonight, we have two small polish changes going in that we’re playing with.

A more appealing comment dialog

The first is a softer, less boxy-feeling comment dialog. Here’s how it looked before:

Old comment dialog

Here’s how it looks now.

New comment dialog in Review Board 2.5

We’ve softened the borders, added some rounded corners, moved the dirty state indicator into the title, and added a slight drop shadow to help it pop. We also made it a bit larger by default, giving you more room to write in.

Again, nothing revolutionary, but nicer to look at.

A less bold chunk highlighter

When navigating through chunks of a diff, Review Board would highlight the selected chunk by surrounding it with 2px-wide black borders. While effective, this was a bit of an eye-sore.

Old diff chunk highlighter

We’re trying something a bit more subtle, yet still quite visible, and cleaner.

New diff chunk highlighter in Review Board 2.5

We’ll still be tweaking all this, and we’re not aiming for any new massive changes in 2.5 at this point (2.6 and 3.0 may have some more refinements/redesigns though). Our aim is to make 2.5 just feel a lot more comfortable and clean, compared to 2.0.

Read More

A new stopwatch extension for Review Board

In any business setting, it’s nice to measure as many data points as possible in order to optimize processes. One common request we’ve had over the years has been the ability to know how much time people are spending on code reviews. This can be used to identify problems on both ends of the spectrum. If people are knocking out reviews in just a few seconds, they’re probably not giving the code the attention it needs to find the defects. Likewise, if code reviews are eating up too much time, it can point to problems of accumulated technical debt or the need for training or mentoring.

Stopwatch Extension

We’re working on a new extension that adds a “stopwatch” to the bottom-right of a review request. This can be toggled on and off, and the total time will be recorded in the review. This data is also available in the API for use by external tools, and we’ll be adding some nice reporting of this data to Power Pack.

Stopwatch Extension - Review header

Read More

Review Board now plays nice with inaccessible SVN commits

If using ACLs to limit access to parts of a Subversion tree, both the New Review Request page and the commits API would break. Inaccessible commits appear in the commit log, but with no data other than a revision. No date, no message, no author, and we expected at least a date to appear.

So that’s fixed up now. Such commits are now shown in the commits list, though with minimal information. Ideally, we’d completely filter these out when querying for them, but that doesn’t seem doable. At least things aren’t totally broken now!

Read More

Djblets now has a Travis-CI

We used to have a whole, internal automated build system that tested Review Board, Djblets, Django Evolution, etc. against a variety of Python and Django versions. This was responsible for automated tests, nightly builds, website doc generation, and so on.

We’re getting this going again, but this time we’re using Travis-CI. To start off, we’re testing this with Djblets. The testing is minimal, but at least it’s there, and we’ll expand it as we go. Here it is.

Read More

Extensions can now reference static assets from Djblets/Review Board

I’ve made some fixes to the extension packaging today that allow stylesheets in an extension to reference assets (images, fonts, other stylesheets) found in Review Board or Djblets. This wasn’t working before, due to some assumptions Django makes when it comes to post-processing and packaging of static files.

Django allows you to define a list of static namespaces with a given name, and any static path starting with that name is looked up in that resource. So, djblets/images/<blah> or rb/fonts/<blah>.

Works great in templates and everywhere else, except for static file packaging. There, the assumption appears to be that any referenced resources live in the same app. The paths built are always relative to the directory or the top of the static namespace, making it impossible to do the kind of references an extension author would want.

Well, almost impossible. I pretty much beat that into submission. And here it is: https://reviews.reviewboard.org/r/7437/

Read More

Converting database evolutions to Django migrations

May have made some progress toward this today…

When you upgrade Review Board, sometimes it needs to modify your database. This will happen if we’ve added new fields, or tables, or renamed a field, or deleted one, or, you know, databasey things like that. This sort of modification is a database schema migration, and they must be done carefully. They also must support all the databases we support (MySQL, PostgreSQL, SQLite).

Historically, for us, this has been done by Django Evolution. Back before Django 1.7, there was no support for database schema migrations, but some clever people came to the rescue with Django Evolution. We’ve been using it since Review Board 1.0 alpha, and for the past several years, we’ve been maintaining it.

Here’s what it does, in a nutshell:

  • It lets us define what changes need to be made in a database-agnostic way, through “evolution files.”
  • It figures out what changes have already been applied (through a table that tracks those changes) and what changes still need to be applied.
  • It then converts those evolution files into a set of mutation operations, optimizes them, and then converts them to SQL statements suitable to the database.

Now, Django 1.7 finally has support for migrations, but of course the format is wildly different. For in-house projects, this is no big deal. You update the database one last time with Django Evolution, then use Django’s migrations from there on out. That doesn’t work with Review Board, though, since any user can go from any release to any release.

We cannot ever upgrade our Django dependency without solving this problem. Oh yeah, and Review Board extensions are affected as well.

So how do we handle this? Well, this is the game plan:

  • Write a converter to convert the evolution files to Django migration files.
  • Add a hook just before Django’s migration attempt, that imports the Django Evolution history into Django migration’s format. It would do this for any unimported apps, so we need to track those.
  • Write some compatibility shims that convert evolution commands to matching Django migration commands, and begin a deprecation phase.

Seems sorta straight-forward, right? Well, this stuff is always more complicated than it should be, so we’ll see.

I’m on step 1, and so far, this is looking like it’ll work. Expect some long posts with descriptions of how gracefully I’m bashing my head into the wall shortly.

Read More

Djblets now mostly works with Django 1.8!

Djblets is our Django utility library full of reusable components for Review Board. It has datagrids (which powers the dashboard), the API framework, the extension framework, dynamic site configuration, and lots of other utilities.

We just landed a set of changes to bring compatibility with Django 1.8! Let me tell you, going from 1.6 to 1.8 is a huge process, and retaining compatibility with both is not easy. Still, this is a big step.

That’s going into Djblets 0.9, but experimentally. We may actually lock Djblets 0.9 to Django 1.6.x, in order to prevent scenarios where a user upgrades Djblets and it accidentally pulls in the wrong version of Django. We’ll see, though

Also, this does not mean Review Board is anywhere near supporting Django 1.8. That’s a loooong ways off. Review Board 3.0 at the earliest. There’s a lot of work to do there still, and a lot of work to do in Django Evolution to convert to Django’s new migrations support.

(That part is scary.)

Read More

The road to Review Board 2.5

f you’ve been following our development for a while, we’ve been working on a big 2.5 release. This started off as a nice little feature release that was going to focus on API tokens, Webhooks, and a handful of other features, but has since evolved to include a UI refresh and preliminary mobile support.

It’s taken longer than we’d like, surely. That happens in software, and we’re still a pretty small team with very full plates. The good news is, we’re getting pretty close to release.

So here’s where we are now:

  • We’re feature-frozen, but still polishing things.
  • We’re fixing bugs here and there (though things are looking quite stable so far).
  • Getting ready to release beta 2 (soon!)
  • Followed by a RC release, and then the final 2.5!

Awesome. So, we’re getting there. Follow us on our ChangeLog, and you’ll get to watch as we get closer and closer to the final 2.5 release.

Read More

Better branch navigation with git-j

One of Git’s core strengths are its simple, light-weight branches. Spend any time working with Git and you’ll soon develop the habit of creating feature branches for any change. As your project grows, you may start to introduce release branches, hotfix branches, and long-running feature branches.

Navigating between branches becomes a common task. A simple git checkout <branchname> is all it takes. Not so bad, but if you’re frequently switching between branches, all the typing can add up.

That’s where git-j comes in.

git j is not only faster to type than git checkout, but it also offers some quick shortcuts for fast branch navigation, making it just as easy as directory navigation.

Here, I’ll show you.

Go up a branch with git j ..

Much like directory navigation, git j .. jumps to the nearest parent branch.

Let’s say you have this branch scheme:

o [topicB] [HEAD]
|
o [topicA]
|
o [master]
|

Say you want to go down a level to topicA, do some work, and then go down again to master. Using git j ..:

$ git j ..
# Do some work
$ git j ..

Now you’re on master. Check out the savings compared to git checkout:

$ git checkout topicA
# Do some work
$ git checkout master

Hey look, you’ve saved 22 keystrokes!

Jump to your previous branch with git j -

Ever find yourself switching back-and-forth between two branches? We have a handy shortcut for that.

Let’s go back to our branch scheme from before. You’re on topicB, and you need to go down a branch to topicA, do some work, and jump back to topicB.

Simple.

$ git j ..
# Do some work
$ git j -

Much nicer than the alternative:

$ git checkout topicA
# Do some work
$ git checkout topicB

That’s another 23 keystrokes saved.

Use aliases for common branches

You probably have a few branches you’re frequently on, and I bet one of them is master. Let’s say that another is release-1.0.x. Oh, let’s also throw in something like 2.0/big-refactor on top of that.

How often are you typing those branch names? Every typo any of them? I sure have.

We can shorten all those names with git j alias. Let’s give them nice, short, easy-to-remember names:

$ git j alias m=master
$ git j alias 1.0=release-1.0.x
$ git j alias rf=2.0/big-refactor`

Now instead of typing those branch names, all you have to do is pass the aliases to git j, like so:

$ git j m
$ git j 1.0
$ git j rf

Much nicer than:

$ git checkout master
$ git cehckout release-1.0.x
$ git checkout 2.0/big-refactor

Wouldn’t you say?

I use that m alias all the time, since practically all Git repositories have a master branch. Instead of setting up an alias for all of them, I can make a global alias:

$ git j alias -g m=master

Now git j m will work wherever I go!

To unset an alias, just pass an empty branch name:

$ git j alias rf=

By the way, if you ever need to check which aliases you’ve set up, just simply type git j aliases. For global aliases, git j aliases -g.

Working with branch history

git j keeps track of which branches you’ve checked out most recently, and makes it easy to jump between them.

Simply run git j history to see what your branch history looks like.

Want to quickly jump to a branch in your history? git j <number> is all you need. For instance, git j 2 will jump 2 branches back in your history.

Putting it all together

Okay, here’s a big, real-world-ish example. Our Review Board repository has master and release-2.0.x branches, and I’ve introduced a my-feature branch. In this example, I’m going to:

  1. Work on release-2.0.x.
  2. Rebase my-feature on top of it.
  3. Merge it into release-2.0.x.
  4. Merge the latest release-2.0.x changes into master.
  5. Go back to working on release-2.0.x.

First, here’s how you’d traditionally do this with Git:

# 1. Work on release-2.0.x.
$ git checkout release-2.0.x

# 2. Rebase my-feature on top of it.
$ git checkout my-feature
$ git rebase release-2.0.x

# 3. Merge it into release-2.0.x.
$ git checkout release-2.0.x
$ git merge my-feature

# 4. Merge the latest release-2.0.x changes into master.
$ git checkout master
$ git merge release-2.0.x

# 5. Go back to working on release-2.0.x.
$ git checkout release-2.0.x

Now let’s try it with git j, and with a couple of aliases (m for master, 2.0 for release-2.0.x):

# 1. Work on release-2.0.x.
$ git j 2.0

# 2. Rebase my-feature on top of it.
$ git j my-feature
$ git rebase release-2.0.x

# 3. Merge it into release-2.0.x.
$ git j ..
$ git merge my-feature

# 4. Merge the latest release-2.0.x changes into master.
$ git j m
$ git merge release-2.0.x

# 5. Go back to working on release-2.0.x.
$ git j -

How’s that for less typing? As you start to work with git-j, it’ll all start feeling more natural, just like walking a filesystem.

Get started with git-j today!

Simply clone our dev-goodies repository and stick dev-goodies/bin/git-j somewhere in your path. Or, stick all of dev-goodies/bin/ in your path to get easy access to all our scripts.

We’ll talk more next week about another extremely useful git script we provide called git-rebase-chain. Stay tuned!

Read More