Work toward a Django 1.8+ port for Review Board

We’ve been dependent on Django 1.6 for our components, for many reasons. There are a lot of things Review Board has to deal with that most Django apps do not, so Django often regresses us, unintentionally. There are also just design changes in Django that don’t suit shipping products, and we’re often having to work around these changes.

However, the biggest bit is our database migration story. Our codebase depends on Django Evolution for migrations, which has to dive pretty far into the internals of Django for some operations. This is a large bit of work to port.

I’m happy to say that, after spending the day on it, I got surprisingly far toward having Django Evolution working on Django 1.8! It’s not perfect, and there’s not yet a good story for playing nice with Django migrations, but it’s a huge start. It opens the doors to getting a full compatibility story going.

The next question is, are we keeping Django Evolution, or moving to migrations fully? Well, that’s a bigger problem, because we have no control over which version of Review Board (and therefore Django) people are upgrading from, and have to be very careful with how we handle any database migrations.

There are also issues with Django’s migrations just being a lot slower than Django Evolution, to the intentional lack of an optimization step when applying the migrations. This means stupidly-long upgrades for large installs, which won’t work for us. So, we’ll probably stay with Django Evolution until we figure out a decent solution there…

Read More

Lots of UI cleanups in preparation for Review Board 2.5 RC 1

2.5 beta 2 looks to be working pretty well, and we’re working hard to get ready for RC 1. As part of this, we’ve fixed up a number of little UI issues here and there. For instance, login on mobile now works:

 

Mobile Login (Review Board 2.5)

 

As does registration and password resets.

Gravatars are now showing up more reliably in the dashboard. Depending on the settings on the server, these may have been hidden unintentionally. Basically, defaults weren’t being taken into consideration in some calls.

The user page now works properly on mobile, with filters moving to a little menu:

 

Mobile Dashboard (Review Board 2.5)

Mobile Dashboard with Filters (Review Board 2.5)

 

Also, some fixes for visual issues with text and Markdown file attachment review pages.

Read More

On-the-fly syntax highlighting when using Markdown

Review Board 2.0 introduced Markdown support for text fields, and we’ve been iterating on this since. One nice advantage to using Markdown is that it’s really easy to syntax-highlight a code fragment, like:

```python
def foo():
print "oh hi there"
```

When saving the comment, this would appear rendered with some syntax highlighting, same as the diff viewer.

In 2.5, we’re adding on-the-fly syntax highlighting for most popular languages: CoffeeScript, CSS, Go, HTML, JavaScript, Perl, PHP, Python, ReStructuredText, Ruby, Shell Scripts, SQL, XML, and YAML.

That means when you type code in a code block, like above, it will show the syntax highlighting immediately, without having to save.

Now, it’s not perfect. We use a different highlighting engine for rendered content vs. on-the-fly content, and they don’t 100% agree on how things should be styled, but it’s close enough.

We’re gaining this ability through an upgrade of CodeMirror, the widget we use for the text fields. We’re giving 5.5 a try (we previously tried upgrading to 4.2 and had issues, but so far so good with 5.5).

This will all ship with Review Board 2.5 RC 1.

Read More

A new polished issue summary table for review requests

Since Review Board 1.6, we’ve had a table of all open issues (comments that have a task that needs fixing before a change can go in), sitting right below the fields of a review request. This could be filtered by reviewer and by status type, showed summaries of the comments, the date/time the comment was filed or last updated, and the status type.

It was pretty text-heavy, though, and not easy to read at a glance. As such, most people probably ignored the nice filtering abilities and, really, most of the content.

So, I redesigned it.

Here’s how it looked before:

Old issue summary table

And here’s how it’ll look in Review Board 2.5 beta 2:

New issue summary table (Review Board 2.5)

New issue summary table (Review Board 2.5)

It’s even more mobile-friendly:

Mobile issue summary table (Review Board 2.5)

Users will get to use the new table later this week.

Read More

Using Review Board with Amazon CodeCommit

Today, Amazon released their all-new CodeCommit service as part of the Amazon Web Services family. CodeCommit is a Git repository hosting service built for scalability and reliability, helping to securely store encrypted versions of your code, binaries, and configuration related to your products and cloud infrastructure.

They’ve put together a guide on integrating AWS CodeCommit with Review Board that you can follow if you’re wanting to give this service a try. It’ll walk you through deploying a Review Board server, setting up access to CodeCommit, linking your repository, and posting changes for review.

Currently, setup requires maintaining an in-sync clone of your repository on the Review Board server. We’re aiming to work with the CodeCommit team to help bring direct support for hosted CodeCommit repositories to a future release of Review Board and RBCommons.

For more information on getting set up, check out the CodeCommit page and read our guides on configuring Git repositories and our recommended RBTools workflows for Git.

Read More

Some early design notes on Review Board API v3

We’re still working on wrapping up 2.5, and have plans for a small 2.6, but we’re also doing some work on the large 3.0 release. This will feature a lot of things, but one of those things is a major set of improvements on top of our API.

Our API is pretty great, overall. A lot of people use it with great success in quite a number of ways. It’s lacking, though, when it comes to some of the types of queries that can be performed.

Basically, the API is nested a bit too deeply. You can’t really make a query like “Give me all diff comments across all review requests from the past year.” We want that to be possible.

We also want a flexible approach to versioning, to help us going forward. There are a lot of options for this, and some work really well for many APIs, but maybe less so for our needs.

So, I’ve been toying with all this, and put together some initial design notes on API v3.

Nothing here is set in stone. I’m totally open to feedback!

Read More

Some diff viewer polish going into 2.5

In Review Board 2.5, we’re working on polishing up bits of the UI, while at the same time keeping a level of familiarity. We’re not going for a radically changed UI, but rather something that feels more like an evolution of what we had.

So tonight, we have two small polish changes going in that we’re playing with.

A more appealing comment dialog

The first is a softer, less boxy-feeling comment dialog. Here’s how it looked before:

Old comment dialog

Here’s how it looks now.

New comment dialog in Review Board 2.5

We’ve softened the borders, added some rounded corners, moved the dirty state indicator into the title, and added a slight drop shadow to help it pop. We also made it a bit larger by default, giving you more room to write in.

Again, nothing revolutionary, but nicer to look at.

A less bold chunk highlighter

When navigating through chunks of a diff, Review Board would highlight the selected chunk by surrounding it with 2px-wide black borders. While effective, this was a bit of an eye-sore.

Old diff chunk highlighter

We’re trying something a bit more subtle, yet still quite visible, and cleaner.

New diff chunk highlighter in Review Board 2.5

We’ll still be tweaking all this, and we’re not aiming for any new massive changes in 2.5 at this point (2.6 and 3.0 may have some more refinements/redesigns though). Our aim is to make 2.5 just feel a lot more comfortable and clean, compared to 2.0.

Read More

A new stopwatch extension for Review Board

In any business setting, it’s nice to measure as many data points as possible in order to optimize processes. One common request we’ve had over the years has been the ability to know how much time people are spending on code reviews. This can be used to identify problems on both ends of the spectrum. If people are knocking out reviews in just a few seconds, they’re probably not giving the code the attention it needs to find the defects. Likewise, if code reviews are eating up too much time, it can point to problems of accumulated technical debt or the need for training or mentoring.

Stopwatch Extension

We’re working on a new extension that adds a “stopwatch” to the bottom-right of a review request. This can be toggled on and off, and the total time will be recorded in the review. This data is also available in the API for use by external tools, and we’ll be adding some nice reporting of this data to Power Pack.

Stopwatch Extension - Review header

Read More

Review Board now plays nice with inaccessible SVN commits

If using ACLs to limit access to parts of a Subversion tree, both the New Review Request page and the commits API would break. Inaccessible commits appear in the commit log, but with no data other than a revision. No date, no message, no author, and we expected at least a date to appear.

So that’s fixed up now. Such commits are now shown in the commits list, though with minimal information. Ideally, we’d completely filter these out when querying for them, but that doesn’t seem doable. At least things aren’t totally broken now!

Read More

Converting database evolutions to Django migrations

May have made some progress toward this today…

When you upgrade Review Board, sometimes it needs to modify your database. This will happen if we’ve added new fields, or tables, or renamed a field, or deleted one, or, you know, databasey things like that. This sort of modification is a database schema migration, and they must be done carefully. They also must support all the databases we support (MySQL, PostgreSQL, SQLite).

Historically, for us, this has been done by Django Evolution. Back before Django 1.7, there was no support for database schema migrations, but some clever people came to the rescue with Django Evolution. We’ve been using it since Review Board 1.0 alpha, and for the past several years, we’ve been maintaining it.

Here’s what it does, in a nutshell:

  • It lets us define what changes need to be made in a database-agnostic way, through “evolution files.”
  • It figures out what changes have already been applied (through a table that tracks those changes) and what changes still need to be applied.
  • It then converts those evolution files into a set of mutation operations, optimizes them, and then converts them to SQL statements suitable to the database.

Now, Django 1.7 finally has support for migrations, but of course the format is wildly different. For in-house projects, this is no big deal. You update the database one last time with Django Evolution, then use Django’s migrations from there on out. That doesn’t work with Review Board, though, since any user can go from any release to any release.

We cannot ever upgrade our Django dependency without solving this problem. Oh yeah, and Review Board extensions are affected as well.

So how do we handle this? Well, this is the game plan:

  • Write a converter to convert the evolution files to Django migration files.
  • Add a hook just before Django’s migration attempt, that imports the Django Evolution history into Django migration’s format. It would do this for any unimported apps, so we need to track those.
  • Write some compatibility shims that convert evolution commands to matching Django migration commands, and begin a deprecation phase.

Seems sorta straight-forward, right? Well, this stuff is always more complicated than it should be, so we’ll see.

I’m on step 1, and so far, this is looking like it’ll work. Expect some long posts with descriptions of how gracefully I’m bashing my head into the wall shortly.

Read More