On-the-fly syntax highlighting when using Markdown

July 25, 2015December 16, 2015 Christian Hammond

Review Board 2.0 introduced Markdown support for text fields, and we’ve been iterating on this since. One nice advantage to using Markdown is that it’s really easy to syntax-highlight a code fragment, like:

```python
def foo():
print "oh hi there"
```

When saving the comment, this would appear rendered with some syntax highlighting, same as the diff viewer.

In 2.5, we’re adding on-the-fly syntax highlighting for most popular languages: CoffeeScript, CSS, Go, HTML, JavaScript, Perl, PHP, Python, ReStructuredText, Ruby, Shell Scripts, SQL, XML, and YAML.

That means when you type code in a code block, like above, it will show the syntax highlighting immediately, without having to save.

Now, it’s not perfect. We use a different highlighting engine for rendered content vs. on-the-fly content, and they don’t 100% agree on how things should be styled, but it’s close enough.

We’re gaining this ability through an upgrade of CodeMirror, the widget we use for the text fields. We’re giving 5.5 a try (we previously tried upgrading to 4.2 and had issues, but so far so good with 5.5).

This will all ship with Review Board 2.5 RC 1.

A new polished issue summary table for review requests

July 13, 2015December 16, 2015 Christian Hammond

Since Review Board 1.6, we’ve had a table of all open issues (comments that have a task that needs fixing before a change can go in), sitting right below the fields of a review request. This could be filtered by reviewer and by status type, showed summaries of the comments, the date/time the comment was filed or last updated, and the status type.

It was pretty text-heavy, though, and not easy to read at a glance. As such, most people probably ignored the nice filtering abilities and, really, most of the content.

So, I redesigned it.

Here’s how it looked before:

And here’s how it’ll look in Review Board 2.5 beta 2:

It’s even more mobile-friendly:

Users will get to use the new table later this week.

Using Review Board with Amazon CodeCommit

July 9, 2015December 16, 2015 Christian Hammond

Today, Amazon released their all-new CodeCommit service as part of the Amazon Web Services family. CodeCommit is a Git repository hosting service built for scalability and reliability, helping to securely store encrypted versions of your code, binaries, and configuration related to your products and cloud infrastructure.

They’ve put together a guide on integrating AWS CodeCommit with Review Board that you can follow if you’re wanting to give this service a try. It’ll walk you through deploying a Review Board server, setting up access to CodeCommit, linking your repository, and posting changes for review.

Currently, setup requires maintaining an in-sync clone of your repository on the Review Board server. We’re aiming to work with the CodeCommit team to help bring direct support for hosted CodeCommit repositories to a future release of Review Board and RBCommons.

For more information on getting set up, check out the CodeCommit page and read our guides on configuring Git repositories and our recommended RBTools workflows for Git.

Some early design notes on Review Board API v3

July 7, 2015December 16, 2015 Christian Hammond

We’re still working on wrapping up 2.5, and have plans for a small 2.6, but we’re also doing some work on the large 3.0 release. This will feature a lot of things, but one of those things is a major set of improvements on top of our API.

Our API is pretty great, overall. A lot of people use it with great success in quite a number of ways. It’s lacking, though, when it comes to some of the types of queries that can be performed.

Basically, the API is nested a bit too deeply. You can’t really make a query like “Give me all diff comments across all review requests from the past year.” We want that to be possible.

We also want a flexible approach to versioning, to help us going forward. There are a lot of options for this, and some work really well for many APIs, but maybe less so for our needs.

So, I’ve been toying with all this, and put together some initial design notes on API v3.

Nothing here is set in stone. I’m totally open to feedback!

Djblets REST API improvements

July 5, 2015December 16, 2015 Christian Hammond

We maintain a Django application/utility framework called Djblets. Amongst other things, this provides the framework for implementing REST APIs. We use this for the Review Board API.

Review Board adds a utility layer on top of this that supports a bunch of other features, including API tokens, base test cases and utilities for API unit testing, API resource module registration/lazy lookups, and more. These are all pretty useful things, so we’ve decided to move them in to Djblets.

What this means is that, starting in Djblets 0.9, any consumers of the API will be able to offer:

API tokens, with access control defined by a powerful JSON policy definition language.
An easier, more complete way to test APIs.
Helpers to make more complex API codebases manageable.
Utilities for implementing flexible API queries (less than, greater than, etc. for numeric fields, for instance).

This also lets us slim down the Review Board API codebase just a bit, and keep our API support consistent on future projects.

Some diff viewer polish going into 2.5

June 26, 2015December 16, 2015 Christian Hammond

In Review Board 2.5, we’re working on polishing up bits of the UI, while at the same time keeping a level of familiarity. We’re not going for a radically changed UI, but rather something that feels more like an evolution of what we had.

So tonight, we have two small polish changes going in that we’re playing with.

A more appealing comment dialog

The first is a softer, less boxy-feeling comment dialog. Here’s how it looked before:

Here’s how it looks now.

We’ve softened the borders, added some rounded corners, moved the dirty state indicator into the title, and added a slight drop shadow to help it pop. We also made it a bit larger by default, giving you more room to write in.

Again, nothing revolutionary, but nicer to look at.

A less bold chunk highlighter

When navigating through chunks of a diff, Review Board would highlight the selected chunk by surrounding it with 2px-wide black borders. While effective, this was a bit of an eye-sore.

We’re trying something a bit more subtle, yet still quite visible, and cleaner.

We’ll still be tweaking all this, and we’re not aiming for any new massive changes in 2.5 at this point (2.6 and 3.0 may have some more refinements/redesigns though). Our aim is to make 2.5 just feel a lot more comfortable and clean, compared to 2.0.

Review Board now plays nice with inaccessible SVN commits

June 22, 2015December 16, 2015 Christian Hammond

If using ACLs to limit access to parts of a Subversion tree, both the New Review Request page and the commits API would break. Inaccessible commits appear in the commit log, but with no data other than a revision. No date, no message, no author, and we expected at least a date to appear.

So that’s fixed up now. Such commits are now shown in the commits list, though with minimal information. Ideally, we’d completely filter these out when querying for them, but that doesn’t seem doable. At least things aren’t totally broken now!

Djblets now has a Travis-CI

June 19, 2015December 16, 2015 Christian Hammond

We used to have a whole, internal automated build system that tested Review Board, Djblets, Django Evolution, etc. against a variety of Python and Django versions. This was responsible for automated tests, nightly builds, website doc generation, and so on.

We’re getting this going again, but this time we’re using Travis-CI. To start off, we’re testing this with Djblets. The testing is minimal, but at least it’s there, and we’ll expand it as we go. Here it is.

Extensions can now reference static assets from Djblets/Review Board

June 18, 2015December 16, 2015 Christian Hammond

I’ve made some fixes to the extension packaging today that allow stylesheets in an extension to reference assets (images, fonts, other stylesheets) found in Review Board or Djblets. This wasn’t working before, due to some assumptions Django makes when it comes to post-processing and packaging of static files.

Django allows you to define a list of static namespaces with a given name, and any static path starting with that name is looked up in that resource. So, djblets/images/<blah> or rb/fonts/<blah>.

Works great in templates and everywhere else, except for static file packaging. There, the assumption appears to be that any referenced resources live in the same app. The paths built are always relative to the directory or the top of the static namespace, making it impossible to do the kind of references an extension author would want.

Well, almost impossible. I pretty much beat that into submission. And here it is: https://reviews.reviewboard.org/r/7437/

Converting database evolutions to Django migrations

June 17, 2015December 16, 2015 Christian Hammond

May have made some progress toward this today…

When you upgrade Review Board, sometimes it needs to modify your database. This will happen if we’ve added new fields, or tables, or renamed a field, or deleted one, or, you know, databasey things like that. This sort of modification is a database schema migration, and they must be done carefully. They also must support all the databases we support (MySQL, PostgreSQL, SQLite).

Historically, for us, this has been done by Django Evolution. Back before Django 1.7, there was no support for database schema migrations, but some clever people came to the rescue with Django Evolution. We’ve been using it since Review Board 1.0 alpha, and for the past several years, we’ve been maintaining it.

Here’s what it does, in a nutshell:

It lets us define what changes need to be made in a database-agnostic way, through “evolution files.”
It figures out what changes have already been applied (through a table that tracks those changes) and what changes still need to be applied.
It then converts those evolution files into a set of mutation operations, optimizes them, and then converts them to SQL statements suitable to the database.

Now, Django 1.7 finally has support for migrations, but of course the format is wildly different. For in-house projects, this is no big deal. You update the database one last time with Django Evolution, then use Django’s migrations from there on out. That doesn’t work with Review Board, though, since any user can go from any release to any release.

We cannot ever upgrade our Django dependency without solving this problem. Oh yeah, and Review Board extensions are affected as well.

So how do we handle this? Well, this is the game plan:

Write a converter to convert the evolution files to Django migration files.
Add a hook just before Django’s migration attempt, that imports the Django Evolution history into Django migration’s format. It would do this for any unimported apps, so we need to track those.
Write some compatibility shims that convert evolution commands to matching Django migration commands, and begin a deprecation phase.

Seems sorta straight-forward, right? Well, this stuff is always more complicated than it should be, so we’ll see.

I’m on step 1, and so far, this is looking like it’ll work. Expect some long posts with descriptions of how gracefully I’m bashing my head into the wall shortly.