ChangeLog: November 5, 2018

The World Update: Python 3 and Django 2

In a previous post, we talked the work we’re doing to move to modern versions of Python and Django. We’re calling this the “World Update,” and it’s quite a large and difficult project.

Database Migrations and Django Evolution

Much of the difficulty comes from an incompatibility between how we handle database migrations (using Django Evolution) and how Django handles them (using their built-in migrations support). These aren’t at all compatible, and for many reasons (legacy support and performance), we don’t want to ditch Django Evolution. This has by far been the biggest hurdle to overcome for the World Update.

We’ve been working on this, reworking nearly all of Django Evolution to better utilize migration support in Django, to provide a new set of generic structures for describing database state and modifications, and to (soon) hook into and partly override the migration process to allow both to co-exist.

Progress on this had been steady, and we feel we’re reaching a point where we’ll be able to fully support migrations in Django Evolution.

Djblets and Review Board Work

We now have Djblets working with modern Python 3.x releases and Django 1.6 through 2.1, so we’re in good shape there.

Review Board has a large number of patches in the works to bring Python 3.x and Django 2.1 compatibility. The biggest challenge in Review Board has to do with string type safety. In Python 2.x, Unicode and byte strings are somewhat interchangeable, but there’s a sharp divide in Python 3. We’ve found many cases where we’ve made bad assumptions about string types in Review Board, and are working on both fixing these and adding strict type checks.

World Update Releases

We’re expecting that Review Board 5.0 will require Python 3.5+ and the latest Django 2.x. Administrators will have to make sure they’re running a modern OS with the necessary version of Python, and extension authors will have some updates to perform, but in the end this will be so worth it.

Review Board 4.0 Status

We were initially hoping to release this in August, but that certainly didn’t happen. Too many things came up over this year that took away from the release.

The good news is, DVCS works, and we’re trying to polish up the experience. There’s a few other features we’re also trying to get into 4.0, but once we’re at a point where we feel people can begin safely using DVCS, the beta will start shipping. That will be… soon-ish. We’ll see what happens with the holidays.

Read More

A New Era of Privacy – Beanbag, Review Board, and You

If you’ve been on the Internet at all in the past few weeks, you’ve likely been bombarded by e-mails from every service you’ve ever used telling you that they’ve updated their privacy policy.

Yes, we’ve updated ours too, but we want to talk to you about how we manage your data, the greater picture of privacy on the Internet, the European Union’s new GDPR legislation, and why this all matters to you, no matter where you are in the world.

GDPR: A new standard for privacy

On May 25th, the GDPR (General Data Protection Regulation) goes into effect in the EU. It gives users there an unprecedented level of control and insight into their personal information. Amongst other things:

  • People must be able to change, delete, or request copies of their personally identifiable information
  • Companies need a valid legal basis for the usage of that information (which may require getting consent for that use)
  • Handing that data off to third-parties also requires a legal basis, and must be documented in a privacy policy
  • That privacy policy must be able to be read and understood without a law degree

You must admit, that’s pretty nice. Though companies are not required to give these rights to non-EU residents, many (including us) are treating this as a new global standard.

Some parts of the GDPR are a bit vague and not all companies see eye-to-eye on the level of control you should have. We’re hoping our approach goes above and beyond.

Our new privacy guarantees

We’ve always collected as little data as needed. We don’t need much, except to provide services to you, to aid in team communication, and to make use of third-party services we trust who help us run our business and provide support to you.

Still, under the GDPR, there was more for us to do. So here’s what we’re promising:

  • We’ll continue to only collect what we strictly need, and to document it clearly in our Privacy Policy.
  • We’ll continue to give you control of your data, and handle deletion and alteration requests, as we always have.
  • We’ve updated our services to request your consent (and give you full control over it at any time) for any optional usage of your personal information, and any usage we strictly require to run our services effectively will be clearly documented.
  • We’ve never sold your information and never will.
  • We’re extending the rights granted by the GDPR to all users of our products, everywhere.
  • If you ever have any questions or concerns about your data, we’re always here to help.

To help, we’ve built a whole new privacy-focused framework in Djblets to help with privacy guarantees and consent requests. All our software will be using this and we’ll be encouraging Review Board extension authors to use it. We’ll talk about this in more detail in an upcoming post.

What to expect by May 25th

Our Privacy Policy is up now, and will take effect on May 25th, 2018.

We’ll be activating the enhanced privacy support on RBCommons, reviewboard.org, reviews.reviewboard.org, and Splat in time for the 25th. If you’re a user on these, the next time you connect you’ll be asked to accept the Privacy Policy and to allow or block usage of your information for some services.

We’ll also be releasing Review Board 3.0.7 and Djblets 1.0.6, which are privacy-enhanced, optionally allowing for Terms of Service and Privacy Policy URLs and GDPR-compliant consent functionality. Many servers may not need this, but it’ll be available for those that do.

If you want to change, delete, or request any of your personal information from our servers, or want more information on all this, reach out to us at any time and we’ll help. You don’t need to wait for May 25th.

Read More

ChangeLog: Catching Up

It’s been too long since we last ran the ChangeLog series, and felt it was the right time to start it back up again. ChangeLog is a look into the latest behind-the-scenes work going into Review Board, RBCommons, and other Beanbag projects. While intended to be a weekly series, we’d like to start off with some of the bigger tasks and feature development from the past month.

Moving to Django 1.11 and Python 3

Today, all current versions of Review Board depend on Django 1.6, an old release that’s no longer supported by the Django project but is by us, and doesn’t support modern Python 3 releases.

We’ve been stuck on 1.6 because 1.7 introduced (and later mandated) a new way of handling database migrations, which is incompatible with the method we’ve always used. Reconciling the differences has been a challenge.

In the past month, we’ve made significant progress toward both the Django and Python updates:

  • Djblets 2.0 (our development release) is now compatible with Django 1.6 through 1.11 and Python 2.7, 3.4, 3.5, and 3.6.
  • Django Evolution (used for database migrations) now works with Django 1.6 through 1.11 and Python 2.7, 3.4, 3.5, and 3.6. Work’s being done to let it co-exist with Django migrations now.
  • Review Board has started receiving patches for Django 1.11 and Python 3.5+ now. This is still in development, and likely won’t make the Review Board 4.0 release, but will be there for 5.0.
  • RBTools 1.0 (shipping in a few months) now has full Python 3 support.

New Release Schedules


We’ve began moving to a train model for releases, and have all of our main and upcoming products now on the calendar.

Here’s what this currently looks like:


  • Review Board 4.0 (with DVCS support!) is expected to ship in August, 2018
  • Review Board 3.0.x releases will ship (generally) every other Tuesday
  • RBTools 1.0 is expected to ship April 12th
  • RB-Gateway is expected to ship March 28th

We’re planning to release a new major Review Board release every ~6 months, meaning smaller but more frequent releases. We’re still experimenting with the schedule and timeframe for these releases.

RB-Gateway

We’ve releasing RB-Gateway 1.0 this month. This is a microservice designed to sit in front of a Git or Mercurial repository, providing an API and set of integrations that can be used by Review Board or any other tool or service for more deeply working with your repository.

RB-Gateway doesn’t change your workflow, and can be dropped in with minimal effort. It completely replaces the cgit/gitweb workaround for standalone Git repositories, and means you don’t need to set up something more complicated like GitLab just to work with Review Board.

You’ll see more information on RB-Gateway’s capabilities when we release later this month, and we’ll cover improvements being made to it here.

Wrapping Up…

Those are really just the major highlights, to get everyone up to speed. It doesn’t include the new features we’ve recently built, like being able to filter files in the diff viewer based on filename patterns, a new command for creating Review Board extension source trees, the work done on kgb, or the crazy investigation into deadlocks that’s delayed Review Board 3.0.4.

Going forward, these will be smaller, covering only what’s been done over the past week. If you like these posts, and want to see this continue, please let us know! You can find us on reddit or on the community support list.

Read More

Some early design notes on Review Board API v3

We’re still working on wrapping up 2.5, and have plans for a small 2.6, but we’re also doing some work on the large 3.0 release. This will feature a lot of things, but one of those things is a major set of improvements on top of our API.

Our API is pretty great, overall. A lot of people use it with great success in quite a number of ways. It’s lacking, though, when it comes to some of the types of queries that can be performed.

Basically, the API is nested a bit too deeply. You can’t really make a query like “Give me all diff comments across all review requests from the past year.” We want that to be possible.

We also want a flexible approach to versioning, to help us going forward. There are a lot of options for this, and some work really well for many APIs, but maybe less so for our needs.

So, I’ve been toying with all this, and put together some initial design notes on API v3.

Nothing here is set in stone. I’m totally open to feedback!

Read More

Djblets REST API improvements

We maintain a Django application/utility framework called Djblets. Amongst other things, this provides the framework for implementing REST APIs. We use this for the Review Board API.

Review Board adds a utility layer on top of this that supports a bunch of other features, including API tokens, base test cases and utilities for API unit testing, API resource module registration/lazy lookups, and more. These are all pretty useful things, so we’ve decided to move them in to Djblets.

What this means is that, starting in Djblets 0.9, any consumers of the API will be able to offer:

  • API tokens, with access control defined by a powerful JSON policy definition language.
  • An easier, more complete way to test APIs.
  • Helpers to make more complex API codebases manageable.
  • Utilities for implementing flexible API queries (less than, greater than, etc. for numeric fields, for instance).

This also lets us slim down the Review Board API codebase just a bit, and keep our API support consistent on future projects.

Read More

Djblets now has a Travis-CI

We used to have a whole, internal automated build system that tested Review Board, Djblets, Django Evolution, etc. against a variety of Python and Django versions. This was responsible for automated tests, nightly builds, website doc generation, and so on.

We’re getting this going again, but this time we’re using Travis-CI. To start off, we’re testing this with Djblets. The testing is minimal, but at least it’s there, and we’ll expand it as we go. Here it is.

Read More

Extensions can now reference static assets from Djblets/Review Board

I’ve made some fixes to the extension packaging today that allow stylesheets in an extension to reference assets (images, fonts, other stylesheets) found in Review Board or Djblets. This wasn’t working before, due to some assumptions Django makes when it comes to post-processing and packaging of static files.

Django allows you to define a list of static namespaces with a given name, and any static path starting with that name is looked up in that resource. So, djblets/images/<blah> or rb/fonts/<blah>.

Works great in templates and everywhere else, except for static file packaging. There, the assumption appears to be that any referenced resources live in the same app. The paths built are always relative to the directory or the top of the static namespace, making it impossible to do the kind of references an extension author would want.

Well, almost impossible. I pretty much beat that into submission. And here it is: https://reviews.reviewboard.org/r/7437/

Read More

Djblets now mostly works with Django 1.8!

Djblets is our Django utility library full of reusable components for Review Board. It has datagrids (which powers the dashboard), the API framework, the extension framework, dynamic site configuration, and lots of other utilities.

We just landed a set of changes to bring compatibility with Django 1.8! Let me tell you, going from 1.6 to 1.8 is a huge process, and retaining compatibility with both is not easy. Still, this is a big step.

That’s going into Djblets 0.9, but experimentally. We may actually lock Djblets 0.9 to Django 1.6.x, in order to prevent scenarios where a user upgrades Djblets and it accidentally pulls in the wrong version of Django. We’ll see, though

Also, this does not mean Review Board is anywhere near supporting Django 1.8. That’s a loooong ways off. Review Board 3.0 at the earliest. There’s a lot of work to do there still, and a lot of work to do in Django Evolution to convert to Django’s new migrations support.

(That part is scary.)

Read More