ChangeLog: New Integrations, Releases, and Prep for RBCommons 3.0

We’ve had a really busy couple of weeks since the last ChangeLog. There were two Review Board releases, a small setback with RB-Gateway, and lots of testing and infrastructure work for RBCommons 3.0.

Review Board 3.0.4 and 3.0.5

Last week, we put out Review Board 3.0.4, a feature-packed release introducing:

It was a pretty great release, fulfilling a lot of feature requests we’ve had for a while an providing the foundation for some new work we’re doing. Unfortunately, there was a last-minute error that, in production, broke part of the form for repository configuration.

Really embarrassing.

Now, we’ve found most people don’t upgrade the same day that a release goes out (downtime must be scheduled, people are busy, etc.) so we mostly started hearing about it two days later. As soon as we realized the mistake, we quickly got a new release out, Review Board 3.0.5, and put some changes in place to help prevent this sort of last-minute problem from happening again.

The good news is that, in the meantime, we went through and fixed a bunch of bugs that didn’t make the 3.0.4 release, but were ready for 3.0.5. So really, we’re just hoping we can all pretend 3.0.4 was just a pre-release for 3.0.5 now ūüôā

Review Board 3.0.6 is currently scheduled for April 10th. I’m expecting it to go smoothly.

RB-Gateway Difficulties and Delays

RB-Gateway, our API wrapper around Git and Mercurial repositories, was supposed to release, well, today. Sadly, that’s not happening.

Let me back up. RB-Gateway is written in Go, unlike most of our projects which are Python-based. Go was chosen partly due to concurrency benefits for handling and serving up requests, and partly for its ease of cross-compilation and distribution (just drop it into a directory and run it on any supported platform).

It’s the cross-compilation that posed a problem. We use git2go, a Go wrapper around libgit2, a C library for talking to Git repositories. We don’t need a lot from it, but it made sense to “go” with that (sorry).

Problem is, including a C library makes cross-compilation much harder, and there’s threads full of discussions on issues with compiling and utilizing git2go in production, depending on how it’s compiled and used. So we’re planning to remove git2go usage.

Instead, we’re evaluating other Git libraries. We probably won’t roll our own, but as we don’t really need much from a Go library, we’ll “go” that route if we need to (sorry).

When that’s done, we should be ready to release.

Prep for RBCommons 3.0

This Friday, we’re beginning an upgrade of RBCommons, bringing many of the features of Review Board 3.0 to the service. We’ve spent much of this week getting this ready — rebuilding servers, testing database migrations, running through checklists of manual feature tests, etc.

There’s going to be a lot to love in this release, but those following Review Board development will surely notice that some features (such as Slack, Asana, etc. integrations) will not be there on launch. We have just a bit more work to do before those are ready. We want those as much as anybody, so they’re high up on the priority list.

The blocker right now is that the administration pages for some of these features are built to plug into the Django administration page, not the custom RBCommons team administration page. So there’s still some work to do before that’s complete. Soon, though!

The upgrade should be smooth, and we should be back up in only a few hours, but just in case, we’re leaving the maintenance window open through Sunday. We aimed for a holiday weekend (well, holiday for a lot of people, anyway) to reduce the impact on users.

Read More

ChangeLog: Catching Up

It’s been too long since we last ran the ChangeLog series, and felt it was the right time to start it back up again. ChangeLog is a look into the latest behind-the-scenes work going into Review Board, RBCommons, and other Beanbag projects. While intended to be a weekly series, we’d like to start off with some of the bigger tasks and feature development from the past month.

Moving to Django 1.11 and Python 3

Today, all current versions of Review Board depend on Django 1.6, an old release that’s no longer supported by the Django project but is by us, and doesn’t support modern Python 3 releases.

We’ve been stuck on 1.6 because 1.7 introduced (and later mandated) a new way of handling database migrations, which is incompatible with the method we’ve always used. Reconciling the differences has been a challenge.

In the past month, we’ve made significant progress toward both the Django and Python updates:

  • Djblets 2.0 (our development release) is now compatible with Django 1.6 through 1.11 and Python 2.7, 3.4, 3.5, and 3.6.
  • Django Evolution (used for database migrations) now works with Django 1.6 through 1.11 and Python 2.7, 3.4, 3.5, and 3.6. Work’s being done to let it co-exist with Django migrations now.
  • Review Board has started receiving patches for Django 1.11 and Python 3.5+ now. This is still in development, and likely won’t make the Review Board 4.0 release, but will be there for 5.0.
  • RBTools 1.0 (shipping in a few months) now has full Python 3 support.

New Release Schedules

‚Ä®We’ve began moving to a train model for releases, and have all of our main and upcoming products now on the calendar.

Here’s what this currently looks like:‚Ä®

  • Review Board 4.0 (with DVCS support!) is expected to ship in August, 2018
  • Review Board 3.0.x releases will ship (generally) every other Tuesday
  • RBTools 1.0 is expected to ship April 12th
  • RB-Gateway is expected to ship March 28th

We’re planning to release a new major Review Board release every ~6 months, meaning smaller but more frequent releases. We’re still experimenting with the schedule and timeframe for these releases.

RB-Gateway

We’ve releasing RB-Gateway 1.0 this month. This is a microservice designed to sit in front of a Git or Mercurial repository, providing an API and set of integrations that can be used by Review Board or any other tool or service for more deeply working with your repository.

RB-Gateway doesn’t change your workflow, and can be dropped in with minimal effort. It completely replaces the cgit/gitweb workaround for standalone Git repositories, and means you don’t need to set up something more complicated like GitLab just to work with Review Board.

You’ll see more information on RB-Gateway’s capabilities when we release later this month, and we’ll cover improvements being made to it here.

Wrapping Up…

Those are really just the major highlights, to get everyone up to speed. It doesn’t include the new features we’ve recently built, like being able to filter files in the diff viewer based on filename patterns, a new command for creating Review Board extension source trees, the work done on kgb, or the crazy investigation into deadlocks that’s delayed Review Board 3.0.4.

Going forward, these will be smaller, covering only what’s been done over the past week. If you like these posts, and want to see this continue, please let us know! You can find us on reddit or on the community support list.

Read More

Announcing virtualenv-multiver for Python Development

If you’ve worked with virtualenvs for Python before for development/testing, then you may have hit cases where you really wanted multiple versions of Python installed in your virtualenv. Which, you may actually have working, because virtualenv, in theory, supports this. In fact, you’re supposed to be able to do:

$ virtualenv -p python2.6 my-env
$ virtualenv -p python2.7 my-env

That’d be great, if it always worked. It doesn’t. When your virtualenv gets built,¬†bin/python¬†may end up being a link to¬†bin/python2.7¬†(or what have you), or it may be the contents instead of a link. Subsequent installs may end up overwriting binaries, producing a¬†python2.6¬†and¬†python2.7that are both Python 2.7.

Oh and it gets worse. On Mac, with a standard Python install, these binaries actually end up invoking ../.Python, a symlink pointing to the system Python. This link is not versioned. So much for multiple Python versions in one virtualenv on the Mac.

A solution!

We fixed this. Now you can run a single command to get a working environment going, without messing with things or running into problems on the Mac. This is virtualenv-multiver.

Now, setting up an environment is as simple as:

$ pip install virtualenv-multiver
$ virtualenv-multiver my-env 2.6 2.7

Couldn’t be easier. This works both for new environments and existing ones.

This is a beta, so there may be some issues here or there. If this is useful to you, give it a try and let us know!

Read More

Work toward a Django 1.8+ port for Review Board

We’ve been dependent on Django 1.6 for our components, for many reasons. There are a lot of things Review Board has to deal with that most Django apps do not, so Django often regresses us, unintentionally. There are also just design changes in Django that don’t suit shipping products, and we’re often having to work around these changes.

However, the biggest bit is our database migration story. Our codebase depends on Django Evolution for migrations, which has to dive pretty far into the internals of Django for some operations. This is a large bit of work to port.

I’m happy to say that, after spending the day on it, I got surprisingly far toward having Django Evolution working on Django 1.8! It’s not perfect, and there’s not yet a good story for playing nice with Django migrations, but it’s a huge start. It opens the doors to getting a full compatibility story going.

The next question is, are we keeping Django Evolution, or moving to migrations fully? Well, that’s a bigger problem, because we have no control over which version of Review Board (and therefore Django) people are upgrading from, and have to be very careful with how we handle any database migrations.

There are also issues with Django’s migrations just being a lot slower than Django Evolution, to the intentional lack of an optimization step when applying the migrations. This means stupidly-long upgrades for large installs, which won’t work for us. So, we’ll probably stay with Django Evolution until we figure out a decent solution there…

Read More

Plans for Review Board’s bug tracker

Many of you may have heard that Google Code is going read-only starting tomorrow, and some have asked us how this will affect the project, since we host our bug tracker there.

Not to worry. Google’s been nice enough to whitelist us for a little while, so even though most of Google Code will be down, we’ll continue to be up. This is not permanent, but for the time-being, you’ll still be able to report bugs at the old address.

Going forward, we’ll be migrating off of Google Code and onto a new tracker. That will happen in the coming weeks, and we’ll talk more about it when it happens.

So why the delay? Why did Google need to extend the shutdown date for us? We actually have something new on the way that we’re pretty excited about. We call it Splat, and while still very young, it’s shaping up to a pretty cool bug/issue tracker. We weren’t quite prepared to switch over to it by the shutdown date, but we have enough of it ready to launch pretty soon.

There’s a lot more that I’d like to say about Splat, but there will be time for that. We’ll make a more formal announcement soon.

Read More

Lots of UI cleanups in preparation for Review Board 2.5 RC 1

2.5 beta 2 looks to be working pretty well, and we’re working hard to get ready for RC 1. As part of this, we’ve fixed up a number of little UI issues here and there. For instance, login on mobile now works:

 

Mobile Login (Review Board 2.5)

 

As does registration and password resets.

Gravatars are now showing up more reliably in the dashboard. Depending on the settings on the server, these may have been hidden unintentionally. Basically, defaults weren’t being taken into consideration in some calls.

The user page now works properly on mobile, with filters moving to a little menu:

 

Mobile Dashboard (Review Board 2.5)

Mobile Dashboard with Filters (Review Board 2.5)

 

Also, some fixes for visual issues with text and Markdown file attachment review pages.

Read More

On-the-fly syntax highlighting when using Markdown

Review Board 2.0 introduced Markdown support for text fields, and we’ve been iterating on this since. One nice advantage to using Markdown is that it’s really easy to syntax-highlight a code fragment, like:

```python
def foo():
print "oh hi there"
```

When saving the comment, this would appear rendered with some syntax highlighting, same as the diff viewer.

In 2.5, we’re adding on-the-fly syntax highlighting for most popular languages: CoffeeScript, CSS, Go, HTML, JavaScript, Perl, PHP, Python, ReStructuredText, Ruby, Shell Scripts, SQL, XML, and YAML.

That means when you type code in a code block, like above, it will show the syntax highlighting immediately, without having to save.

Now, it’s not perfect. We use a different highlighting engine for rendered content vs. on-the-fly content, and they don’t 100% agree on how things should be styled, but it’s close enough.

We’re gaining this ability through an upgrade of CodeMirror, the widget we use for the text fields. We’re giving 5.5 a try (we previously tried upgrading to 4.2 and had issues, but so far so good with 5.5).

This will all ship with Review Board 2.5 RC 1.

Read More

A new polished issue summary table for review requests

Since Review Board 1.6, we’ve had a table of all open issues (comments that have a task that needs fixing before a change can go in), sitting right below the fields of a review request. This could be filtered by reviewer and by status type, showed summaries of the comments, the date/time the comment was filed or last updated, and the status type.

It was pretty text-heavy, though, and not easy to read at a glance. As such, most people probably ignored the nice filtering abilities and, really, most of the content.

So, I redesigned it.

Here’s how it looked before:

Old issue summary table

And here’s how it’ll look in Review Board 2.5 beta 2:

New issue summary table (Review Board 2.5)

New issue summary table (Review Board 2.5)

It’s even more mobile-friendly:

Mobile issue summary table (Review Board 2.5)

Users will get to use the new table later this week.

Read More

Using Review Board with Amazon CodeCommit

Today, Amazon released their all-new CodeCommit service as part of the Amazon Web Services family. CodeCommit is a Git repository hosting service built for scalability and reliability, helping to securely store encrypted versions of your code, binaries, and configuration related to your products and cloud infrastructure.

They’ve put together a guide on¬†integrating AWS CodeCommit with Review Board¬†that you can follow if you’re wanting to give this service a try. It’ll walk you through deploying a Review Board server, setting up access to CodeCommit, linking your repository, and posting changes for review.

Currently, setup requires maintaining an in-sync clone of your repository on the Review Board server. We’re aiming to work with the CodeCommit team to help bring direct support for hosted CodeCommit repositories to a future release of Review Board and¬†RBCommons.

For more information on getting set up, check out the CodeCommit page and read our guides on configuring Git repositories and our recommended RBTools workflows for Git.

Read More