ChangeLog: March 6, 2020 — Nearing Beta

These are exciting times, right? While everyone is preparing for the coronavirus, we’ve been preparing for Review Board 4.0 beta 1.

Review Board 4.0 Beta 1

And the coronavirus.

The shelves are getting pretty empty in my local grocery store, but I’ve kept my sanity.

Preparing for Review Board 4.0 Beta 1

We have an internal target date for 4.0 beta 1, which is coming up very soon, but we’re not prepared to release that just yet. Still a couple factors that might need to delay a week.

This beta will feature:

  • Posting and reviewing multi-commit review requests
  • A brand new, streamlined, mobile-friendly administration UI
  • A full move to Django 1.11, with preliminary Python 3 support (which will affect extensions, so get ready)
  • Read-only mode, letting you prepare for an upgrade without shutting the whole server down
  • A new “Overview” mode in the Dashboard showing all incoming and outgoing review requests
  • A new integration for Jenkins CI
  • Live thumbnails for video file attachments
  • Beginnings of accessibility improvements, improving keyboard input, color contrast, and screen reader integration
  • New JavaScript and CSS components usable by third-party extensions
  • An overhaul of the diff parsers, setting things up for some major future improvements to diffing capabilities (including our in-development new DiffX proposal)
  • Removal of a lot of deprecated internal APIs (again, this may affect extensions)

We’re working on the final bits of the administration UI, and expect to be done by end of next week. Everything is in place, with the exception of the Integrations, Extensions, and Security Center pages. But that’s it!

We’ve also been finishing up the work to make our third-party integrations (Travis-CI, Asana, Jenkins, etc.), Review Bot, and Power Pack all compatible with Review Board 4.0, Django 1.11, and Python 3. These are the last big blockers before release.

After all that is done, beta 1 will be a go!

(Assuming none of us get the coronavirus.)

If you are building custom extensions, please get ready to test this beta. Let us know if you plan to test so we can coordinate with you and give you a heads up on what may change.

RBTools 2.0 Beta

RBTools 2.0 features full support for multi-commit review requests. We’ve had the bulk of this work done for a while, but have been really thoroughly testing it in preparation for a beta, which will go out alongside Review Board 4.0 beta 1.

This week, we’ve finished up the work to apply patches from a multi-commit review request (giving the ability to apply all commits sequentially or squash into a single commit), and to fully land changes.

We’re wrapping this up soon. If you test Review Board 4.0 beta 1, you’ll need RBTools 2.0 beta 1.

What else, what else..

Accessibility Improvements

We’re continuing to work toward improving our accessibility story in Review Board. While we’re nowhere close to where we want to be yet, we’ve amended our policies around new UI components to ensure that keyboard usage and ARIA roles are a first-class citizen in any designs.

If you make use of any assistive technologies, such as screen readers, we’d love to talk to you and get your feedback on a few things!

Video Thumbnails

Review Board 4.0 will be making it easier to glance over file attachments for video files. Just like file attachment thumbnails for images and text files give you a preview of the contents, they’ll soon give you a preview of the video as well.

Hovering the mouse over these thumbnails will cause the video to play (muted) until you move the mouse away. Hovering over them again causes the video to resume from where it left off.

This is great if someone’s using video files to demonstrate their feature. We’re hoping to make this the start of some useful UI around videos.

New UI Components

This past week saw the development of several new accessible CSS/JavaScript components, which can be used by extensions:

New Button Classes

Buttons

We’ve historically had a few different ways to show buttons. Any <input type="button" />, <input type="submit" />, or anything with the .btn CSS class would have a consistent appearance, and could be modified by a set of additional CSS classes.

This has been revised to include <button> (which we oddly did not have before) and anything using the .rb-c-button CSS class (the successor to .btn, which is now deprecated).

Button Groups

Button groups

A button group (.rb-c-button-group in LessCSS) is a collection of buttons, packed together either horizontally or vertically, providing an almost toolbar appearance.

Pop-up Menus

Pop-up menus

The new RB.MenuView (JavaScript) and .rb-c-menu (LessCSS) component manages a pop-up menu that can be shown or hidden as needed. It supports full keyboard navigation and ARIA attributes for accessibility.

We’re working on moving to this in every place that involves a pop-up menu, including review request actions (like “Update” and “Close”) and the Account, Support, and Follow menus in the upper-right of pages.

Menu Buttons

Menu buttons

RB.MenuButtonView (JavaScript) and .rb-c-menu-button (LessCSS) implements a type of button that displays a pop-up menu when clicked.

There are two main ways this can be used:

  1. As a single button that will show some text and a drop-down indicator, used just to display the menu.
  2. As a group of action buttons, all packed together (but visually separated), containing a final button that displays the drop-down menu.

The later is great when you want to offer a common action, but make alternative actions easily available.

Like the new pop-up menus, these provide full keyboard navigation and ARIA attributes for accessibility, and are a great building block for extensions to use.

We hope to provide full documentation at a later time on the standard library of UI components we’re building.

New KGB Release Coming

KGB is our Python module for unit tests, which provides the ability to spy on any function and track the calls and results, or override the behavior. It’s incredibly powerful, and we make heavy of use of it in Review Board and all our other projects.

We’ve just fixed up KGB to work with Python 3.8, so we’re preparing for a release pretty soon.

If you’re a Python developer, check out KGB. Tell your friends. Give us some feature requests. Honestly, it’s pretty complete these days, but we need more reasons to bump the version number.

Alright, that’s enough for this ChangeLog.

Coming Up

We may skip next week’s ChangeLog to focus on finishing beta 1, unless we have anything really interesting to report.

If you want to know more, have any questions, or are curious about anything else, please reach out on our community forum.

We’re also on Reddit (/r/reviewboard), Twitter, Facebook, and YouTube if you want other ways to keep up with Review Board and Beanbag.

Read More

ChangeLog: January 23, 2020 — Student Sprint and Django Upgrade

What a week it’s been. We have two topics to go over: Last weekend’s student mentorship sprint, and a status update on the big Django upgrade.

The student sprint was great!

Last weekend was our student sprint for CANOSP, kindly hosted by the Startup Edmonton in Edmonton, Canada. There, we met up with our new students from the University of Alberta and University of British Columbia, and spent the weekend getting to know them, teaching them about Review Board and how we do things here, and letting them loose on projects.

Students working hard on projects

David giving an architectural overview

David giving an architectural overview (close up)

We may have been sleepy most of the time (these start early!), and VERY cold (had to sit by a stack of ice cubes to keep warm), but everyone had a lot of fun, and have since been hard at work on their projects.

The main focuses for this semester are accessibility and usability. We’re working to make the product easier to use, improving keyboard navigation, and experimenting with ways to offer useful inline help. Much of this work is slated for Review Board 4.0.

Django upgrade time!

This week also marks the end of Django 1.6 in Review Board. We’ve been working for a long time on getting onto a newer version of Django, which has been a much more complex project for us than for most, and we’re finally ready to bump our Django requirement to 1.11.

Diff showing the change to upgrade to Django 1.11

For those paying attention, Django itself is at 3.0.2, but 1.11 is the last version to support Python 2.7. While Python 2.x is now end-of-life’d, that doesn’t mean it’s not in active use in enterprise, and frankly, many of our users are just not ready to upgrade. So Review Board 4.0 will continue to be providing compatibility for 2.7.

By the weekend, we should be on 1.11, and then we’ll be getting ready to test this in production. If all goes well, a beta will follow soon.

If you build extensions for Review Board, you’re going to need to make some changes to support Django 1.11. We have a bunch of useful information on Django updates on our wiki, which we’ll also include in the release notes. Make sure you give the beta a try and begin porting early.

We offer support contracts that cover development assistance, if you need it.

Back to work!

If you have any questions, or anything you’re curious about and want us to cover, please reach out on our community forum.

We’re also on Reddit (/r/reviewboard), Twitter, Facebook, and YouTube (featuring student project videos!) if you want other ways to keep up with what we’re up to.

Read More

ChangeLog: January 16, 2020 — Review Board Status and New Students

Hi everyone. Welcome to the first ChangeLog of 2020! We skipped last week due to the new Review Board 3.0.16 release, so let’s dive in and get caught up.

Review Board 3.0.16

Last Tuesday, we released Review Board 3.0.16. It’s been a long time since 3.0.15, and there are two reasons for this:

  1. Most of our attention of late has been on completing the remaining architectural work on Review Board 4.0 (Python 3 and Django 1.11 porting) and RBCommons user roles and billing updates.
  2. We’ve been trying to carefully design and implement some large backend improvements for repositories and repository configuration, in collaboration with another vendor, and wanted to get it right before we released.

We’ve discussed the repository improvements before, so read that if you want to learn a bit more, but the gist is that we’re giving SCMTools (repository backends) a lot more flexibility in how they present repository configuration, how they’re registered for use in Review Board, and how extra data for repositories get stored. This will lead to some significant improvements in the coming months for a couple of our supported repository types.

Now unless we find some major bug fixes in 3.0.16, it’ll probably be a little while until 3.0.17. We have a backlog of RBTools work we plan to release, and we’re still trying to get 4.0’s architectural rework done.

Review Board 4.0 Status

Most of the rewritten administration UI is in a usable state, and we’re just getting it all ready to be reviewed and landed. After this, we’ll be bumping our Django dependency and performing a bunch of real-world usage tests, just to make sure there isn’t some big breakage some place.

(If you want to learn more about the administration UI work and how it relates to Django and the release, read the ChangeLogs from October 24, 2019, October 31, 2019, and November 7, 2019.)

Once we’re happy, we’ll ship 4.0 Beta 1. Almost there!

New Semester, New Students

Every semester, we take on a batch of CS students eager to work on some real-world project, currently as part of the CANOSP program run by the University of Alberta, Canada.

It all starts this weekend at a get-together in Edmonton, Canada, where we’ll be helping five new students get set up, go over architecture and standards, and start them on their projects.

We’ll talk more about what they’ll be working on next week, but they mostly center around quality-of-life improvements to Review Board.

Wrapping Up The Week

If you have any questions, or anything you’re curious about and want us to cover, please reach out on our community forum.

We’re also on Reddit (/r/reviewboard), Twitter, Facebook, and YouTube (featuring student project videos!) if you want other ways to keep up with what we’re up to.

Read More

ChangeLog: October 31, 2019 — DiffX, CSS Components, Admin UI

This week, we’re going to dive into three major topics:

  • DiffX, our proposed standard for a better Unified Diff format
  • Our new in-house CSS Components standard
  • The work being done on the Review Board Administration UI

DiffX

Unified Diffs, the most common type of diff files, are pretty lacking in standards. Nearly every type of source code management solution outputs its own version of a unified diff, shoving metadata into it however the developers saw fit. This frankly makes it a pain to work with diffs in Review Board, and I’m sure anyone else building similar tools can say the same thing.

A few years ago, we jotted down some ideas of what we’d like to see in diffs. We wanted:

  1. A superset of Unified Diffs that’s fully backwards-compatible (no way would a new diff format gain traction if existing tools couldn’t use it)
  2. A format that could be reliably serialized/deserialized (complete with metadata and file contents) as easily as something like JSON without writing a parser for a dozen different variations of the format
  3. Information on the encodings of a diff file and of the contents of a file, so we no longer have to guess
  4. Changes across multiple commits in a single file
  5. Some standard way of representing changes to binary files

We feel this is important. You wouldn’t believe how wildly different two diff files can be, and how broken most are (just having spaces in a filename means you’re going to run into all sorts of crazy problems).

So we put these ideas together and called it DiffX. Our plan was to build a specification, a standard Python library (and later libraries for other languages), and RBTools/Review Board support.

This week we put up the initial draft of the DiffX specification and documentation. Take a look. See what horrors tools like ours go through, and what we’d like to do about it.

Do you work on another code review tool or source code management system? Let’s talk!

Note that this is still a work-in-progress, and we have a lot we need to better formalize the parsing and the metadata format (and make a public, useful reference implementation).

CSS Components

During Review Board 4.0 development, we kept running into clashes between CSS rules, and had to deeply nest CSS classes and elements in order to ensure our styles would apply. Yuck. This was not sustainable.

We sought to fix this for our code, and looked into what others have done. We considered BEM, BEMIT, and a few other approaches. None felt quite right for our needs (particularly since we want to avoid clashes between third-party extensions).

So we took what we liked in each and drafted our CSS Component Style Guide.

This gives us a better way of thinking about what goes into our stylesheets and HTML pages. We think in terms of formal, reusable components, instead of just haphazardly tacking on styles for whatever HTML we’re writing.

We group together LessCSS variables (used for colors, sizes, etc.) and macros (for customizing a component’s presentation in a safe way) under organized namespaces. Stylesheets (ours and extensions’) that are looking to fit in with our styles or specialize a component can rely on these to better fit in.

We’re also taking care to add formal documentation for everything we add, which forces us to think about our styles, reason about them, and support them.

Want to see this in action? Here’s a few examples:

We’re still working on moving over to this, writing all new CSS using this standard and migrating old styles as needed. This is a long-term project, but we thought it might be interesting to some of you.

If you’re developing extensions for Review Board, we encourage adopting these new styling conventions. It’ll make your life so much easier.

The New Administration UI

Last week, we talked about the work on Python 3 and Django 1.11, and some of the huge tasks we’ve had to work through to support this.

One of the large — and final — hurdles is our administration UI. Django helpfully provides a built-in administration UI that projects get out of the box, and it’s very useful. And we extend and customize it heavily.

The problem is that Django’s administration UI has changed a lot since 1.6, and frankly it wrecked ours. So we’re rebuilding it, better than it was before.

Here’s a breakdown on the new UI:

  • It’s mobile-friendly
  • Organization is now much more clear — all navigation is in the sidebar, not split between sidebar and the top header bar
  • The administration dashboard widgets are so much better — larger, less cluttered, faster, and trimmed down to only the most useful widgets
  • It’s completely our own, no longer dependent on Django’s rendering, so future upgrades will be smoother

It’s also built with our new CSS component standard, and introduces new components that we can use throughout the product. Extensions can take advantage of these, too, helping them look like a native part of Review Board without worrying too much about our styling.

Here’s a couple in-progress screenshots.

Next Week

We’ll go over some of the new diff functionality being worked on in Review Board 4.0, and what we’ve been up to in RBCommons land.

Again, if you have any questions, or anything you’re curious about and want us to cover, please reach out on our community forum.

Read More

ChangeLog: October 24, 2019 – The Python 3 and Django 1.11 Migration

Hi, Christian Hammond here. Welcome back to ChangeLog, where we cover the latest going on in Review Board, RBCommons, and Beanbag in general. It’s been a while since we’ve posted one of these, but we’re aiming to bring this back as a weekly series.

There’s a lot to talk about.

The Short Version

  • We delayed Review Board 4.0 to bring Django 1.11 and Python 3 support. This will not be released by Python 2’s End Of Life date.
  • Python 3 and Django 1.11 support has been in the works for a long time, but is far more complex than it may seem.

The Longer Version: Where are we today

We’ve had four main focuses this year:

  • Getting Review Board 4.0 beta 1 ready to launch. This is way behind our original schedule, but intentionally so — we’ll go over the reasons why in a minute.
  • Bringing integrations options to RBCommons and improving team management, billing, and signup.
  • Building out new functionality for Power Pack (PDF diffing, and in-progress authentication improvements).
  • Growing our business (the prior two tasks were part of that), supporting customers, and assisting some with large projects of their own.

This along with a multitude of smaller tasks taking our time throughout the year (and, on a personal note, dealing with the aftermath of the Paradise, CA “Camp Fire.”)

Most of you are here to find out about Review Board, so let’s dive into that.

The “World Update”: Review Board 4.0, Python 3, and Django 1.11

Our original goal for Review Board 4.0 was to bring support for multi-commit review requests and to ship that. This was a large project, as Review Board was originally written before DVCS was common (Subversion and Perforce were the primarily open source and enterprise solutions).

The problem though, is Python 2 End Of Life is coming up fast, and if we kept on schedule with Review Board 4.0, it would be a long time until we’d support Python 3. So we decided to delay 4.0 in order to get it ready for Python 3.

Note that we also still need to support Python 2.7, since many companies are still heavily tied to 2.7 (older distros, custom extensions and scripts).

So what’s hard about supporting Python 3? Welllll….

The Python Compatibility Problem

Python 3 is leaps and bounds better than Python 2 in most ways, but porting a large and complex product from Python 2 to 3 is even harder than you think.

Most Python users know of the major differences:

  • Modules have moved and functions have been renamed
  • The default string type has changed from byte strings to Unicode
  • Some operators have changed
  • There’s new syntax additions
  • etc.

We thought we prepared years ago to make this move easy. We used Unicode strings in every file. We used the six module to help with using the right modules, types, and functions.

In the end, it was harder than we expected. Our biggest challenge was definitely the Byte Strings vs. Unicode Strings differences. We thought we were in good shape for this, but we weren’t close.

Review Board does a lot of text processing. We’re parsing uploaded diffs, pulling source files from repositories, matching those up and applying the patch to the source, and generating side-by-side diffs. Much of this logic is over 10 years old, even so, we put a substantial amount of work in preparing for Python 3 years ago, so we were shocked by how much we got this wrong.

The problem comes down to the differences in how Python 2 and 3 would handle shoving two different string types together, which is very easy to do accidentally. Python 2 would roll with it, as in many cases the string types were compatible (and would automatically encode/decode so long as the content was basically ASCII). In Python 3, they outright break — which is a better approach, but hard to transition to — and it led us down a rabbit hole of problems.

Missing b'...' prefixes, changes in string return types from Python functions, functions that are no longer compatible with both string types, and very difficult decisions to make regarding compatibility with third-party SCMs.

All the little regressions and inconsistencies added up. Here’s a few more examples:

  • Using Python’s pickle library was a mess. Defaults have changed and pickled data became incompatible (due to module reference and string type changes). We had to build in compatibility layers here and test them thoroughly.
  • Anything that even subtly/unintentionally relied on dictionary sort orders would break.
  • Getting data in/out of processes, streams, and many other objects and functions is now way more sensitive to encoding issues, and required a lot of careful rewrites and testing.
  • Many functions (in Python and third-party modules) that used to return lists now return generators, due to their reliance on other functions that changed return types, and this can cause all sorts of subtle behavioral changes.

We’ve spent a lot of time tracking down issues that might immediately crash or might affect data several stages down, and sometimes be traced to something outside our codebase.

Fixing some Python issues means upgrading third-party modules, which can introduce their own new set of changes, regressions, and new rounds of work.

And the biggest source of that was Django.

The Django Problem

Django 1.6 (the version we’ve been using) breaks on Python 3.6+. This meant we had to upgrade to a newer version, something we’ve been putting off.

Django 1.7 introduced built-in support for database migrations. Quite nice for many projects, but the design was sub-optimal for applications like ours (upgrades could take hours or days longer than our approach) and was fundamentally incompatible with our own Django Evolution.

Many other core components of Django (the foundation for the administration UI, template rendering, URL management, forms, and all sorts of other common and obscure parts) have also largely changed over time in some pretty important ways.

We needed a solution for all of this, and we knew that would take time. To compensate we had to:

  1. Rewrite Django Evolution completely to coordinate evolutions and migrations and support modern Django: ~2 years of work, off and on, with hard problems to ponder
  2. Rewrite our administration UI completely to disconnect from Django’s (ensuring we don’t break again when we move to Django 2.x): estimated ~3 months of steady work, still in progress
  3. Build compatibility layers to help keep older code working and help with new code: ~2 months
  4. Just porting in general, updating dependencies, etc: ~3 months

We’re not done, but we’re close.

The Extensions Problem

Review Board is built to allow third-parties to extend its functionality, modify behavior, and link up with other in-house services. We try to be careful not to break extension functionality, and over time we’ve gotten more strict about providing compatibility and an upgrade path for functionality we want to deprecate or change.

The Python 3 and Django 1.11 upgrade is going to affect just about everybody who’s writing an extension. Some of the work we’ve done on adding compatibility layers will help with this process, but it’s going to mean a longer upgrade cycle for some companies.

We knew from the beginning that we’re going to have our hands full helping these companies out (as part of our support contracts), and that it’d be in our best interest to delay the release and break everything all at once instead of spreading out the breakages across multiple releases, and to also give ourselves time to figure out how best to minimize those breakages.

Most projects moving to Python 3 or to newer versions of Django don’t have to worry about this domino effect in the same way.

Timetable?

I hesitate to say when beta 1 will be done (been wrong before), but I can say that the last major piece of development is wrapping up. We’ll be kicking the tires on it, and want to get a beta out as soon as it’s stable enough.

We may not enable Python 3 builds for beta 1, focusing instead on Django 1.11 testing (Python 3 support is still going to be in development during this time), however we’re working with select customers on real-world testing against Python 3.

Community Questions

Every week, we’d like to address some questions, concerns, suggestions, etc. from the community. If you have any questions for next week, please reach out to us.

Q. The community forum seems quiet at times. Why is that?

A. Support requests from companies with support contracts are conducted over a separate support tracker. We always prioritize these support requests over any other work, and increasingly more companies are moving to this method of support for Review Board.

Q. Will Review Board 4.0 ship with Python 3 before the Python 2 End Of Life date?

A. No, it’s going to miss that date. Python 2.7 is still going to be required for now.

Q. There’s years of open review requests on reviews.reviewboard.org. Why is that?

A. We work with university students every semester to help prepare them for their jobs in the tech industry. They spend their semester building features for Review Board, most of which are prototypes. These make up the majority of the review requests on there. Others are contributions that might be outdated, might have been missed, might be incomplete, or might just not have been reviewed yet.

Next Week

We’ll be going over our new CSS component standard for the project and dive deeper into the new administration UI.

Again, if you have any questions, or anything you’re curious about and want us to cover, please reach out on our community forum.

Read More

ChangeLog: November 5, 2018

The World Update: Python 3 and Django 2

In a previous post, we talked the work we’re doing to move to modern versions of Python and Django. We’re calling this the “World Update,” and it’s quite a large and difficult project.

Database Migrations and Django Evolution

Much of the difficulty comes from an incompatibility between how we handle database migrations (using Django Evolution) and how Django handles them (using their built-in migrations support). These aren’t at all compatible, and for many reasons (legacy support and performance), we don’t want to ditch Django Evolution. This has by far been the biggest hurdle to overcome for the World Update.

We’ve been working on this, reworking nearly all of Django Evolution to better utilize migration support in Django, to provide a new set of generic structures for describing database state and modifications, and to (soon) hook into and partly override the migration process to allow both to co-exist.

Progress on this had been steady, and we feel we’re reaching a point where we’ll be able to fully support migrations in Django Evolution.

Djblets and Review Board Work

We now have Djblets working with modern Python 3.x releases and Django 1.6 through 2.1, so we’re in good shape there.

Review Board has a large number of patches in the works to bring Python 3.x and Django 2.1 compatibility. The biggest challenge in Review Board has to do with string type safety. In Python 2.x, Unicode and byte strings are somewhat interchangeable, but there’s a sharp divide in Python 3. We’ve found many cases where we’ve made bad assumptions about string types in Review Board, and are working on both fixing these and adding strict type checks.

World Update Releases

We’re expecting that Review Board 5.0 will require Python 3.5+ and the latest Django 2.x. Administrators will have to make sure they’re running a modern OS with the necessary version of Python, and extension authors will have some updates to perform, but in the end this will be so worth it.

Review Board 4.0 Status

We were initially hoping to release this in August, but that certainly didn’t happen. Too many things came up over this year that took away from the release.

The good news is, DVCS works, and we’re trying to polish up the experience. There’s a few other features we’re also trying to get into 4.0, but once we’re at a point where we feel people can begin safely using DVCS, the beta will start shipping. That will be… soon-ish. We’ll see what happens with the holidays.

Read More

ChangeLog: Catching Up

It’s been too long since we last ran the ChangeLog series, and felt it was the right time to start it back up again. ChangeLog is a look into the latest behind-the-scenes work going into Review Board, RBCommons, and other Beanbag projects. While intended to be a weekly series, we’d like to start off with some of the bigger tasks and feature development from the past month.

Moving to Django 1.11 and Python 3

Today, all current versions of Review Board depend on Django 1.6, an old release that’s no longer supported by the Django project but is by us, and doesn’t support modern Python 3 releases.

We’ve been stuck on 1.6 because 1.7 introduced (and later mandated) a new way of handling database migrations, which is incompatible with the method we’ve always used. Reconciling the differences has been a challenge.

In the past month, we’ve made significant progress toward both the Django and Python updates:

  • Djblets 2.0 (our development release) is now compatible with Django 1.6 through 1.11 and Python 2.7, 3.4, 3.5, and 3.6.
  • Django Evolution (used for database migrations) now works with Django 1.6 through 1.11 and Python 2.7, 3.4, 3.5, and 3.6. Work’s being done to let it co-exist with Django migrations now.
  • Review Board has started receiving patches for Django 1.11 and Python 3.5+ now. This is still in development, and likely won’t make the Review Board 4.0 release, but will be there for 5.0.
  • RBTools 1.0 (shipping in a few months) now has full Python 3 support.

New Release Schedules


We’ve began moving to a train model for releases, and have all of our main and upcoming products now on the calendar.

Here’s what this currently looks like:


  • Review Board 4.0 (with DVCS support!) is expected to ship in August, 2018
  • Review Board 3.0.x releases will ship (generally) every other Tuesday
  • RBTools 1.0 is expected to ship April 12th
  • RB-Gateway is expected to ship March 28th

We’re planning to release a new major Review Board release every ~6 months, meaning smaller but more frequent releases. We’re still experimenting with the schedule and timeframe for these releases.

RB-Gateway

We’ve releasing RB-Gateway 1.0 this month. This is a microservice designed to sit in front of a Git or Mercurial repository, providing an API and set of integrations that can be used by Review Board or any other tool or service for more deeply working with your repository.

RB-Gateway doesn’t change your workflow, and can be dropped in with minimal effort. It completely replaces the cgit/gitweb workaround for standalone Git repositories, and means you don’t need to set up something more complicated like GitLab just to work with Review Board.

You’ll see more information on RB-Gateway’s capabilities when we release later this month, and we’ll cover improvements being made to it here.

Wrapping Up…

Those are really just the major highlights, to get everyone up to speed. It doesn’t include the new features we’ve recently built, like being able to filter files in the diff viewer based on filename patterns, a new command for creating Review Board extension source trees, the work done on kgb, or the crazy investigation into deadlocks that’s delayed Review Board 3.0.4.

Going forward, these will be smaller, covering only what’s been done over the past week. If you like these posts, and want to see this continue, please let us know! You can find us on reddit or on the community support list.

Read More

Work toward a Django 1.8+ port for Review Board

We’ve been dependent on Django 1.6 for our components, for many reasons. There are a lot of things Review Board has to deal with that most Django apps do not, so Django often regresses us, unintentionally. There are also just design changes in Django that don’t suit shipping products, and we’re often having to work around these changes.

However, the biggest bit is our database migration story. Our codebase depends on Django Evolution for migrations, which has to dive pretty far into the internals of Django for some operations. This is a large bit of work to port.

I’m happy to say that, after spending the day on it, I got surprisingly far toward having Django Evolution working on Django 1.8! It’s not perfect, and there’s not yet a good story for playing nice with Django migrations, but it’s a huge start. It opens the doors to getting a full compatibility story going.

The next question is, are we keeping Django Evolution, or moving to migrations fully? Well, that’s a bigger problem, because we have no control over which version of Review Board (and therefore Django) people are upgrading from, and have to be very careful with how we handle any database migrations.

There are also issues with Django’s migrations just being a lot slower than Django Evolution, to the intentional lack of an optimization step when applying the migrations. This means stupidly-long upgrades for large installs, which won’t work for us. So, we’ll probably stay with Django Evolution until we figure out a decent solution there…

Read More

Extensions can now reference static assets from Djblets/Review Board

I’ve made some fixes to the extension packaging today that allow stylesheets in an extension to reference assets (images, fonts, other stylesheets) found in Review Board or Djblets. This wasn’t working before, due to some assumptions Django makes when it comes to post-processing and packaging of static files.

Django allows you to define a list of static namespaces with a given name, and any static path starting with that name is looked up in that resource. So, djblets/images/<blah> or rb/fonts/<blah>.

Works great in templates and everywhere else, except for static file packaging. There, the assumption appears to be that any referenced resources live in the same app. The paths built are always relative to the directory or the top of the static namespace, making it impossible to do the kind of references an extension author would want.

Well, almost impossible. I pretty much beat that into submission. And here it is: https://reviews.reviewboard.org/r/7437/

Read More

Converting database evolutions to Django migrations

May have made some progress toward this today…

When you upgrade Review Board, sometimes it needs to modify your database. This will happen if we’ve added new fields, or tables, or renamed a field, or deleted one, or, you know, databasey things like that. This sort of modification is a database schema migration, and they must be done carefully. They also must support all the databases we support (MySQL, PostgreSQL, SQLite).

Historically, for us, this has been done by Django Evolution. Back before Django 1.7, there was no support for database schema migrations, but some clever people came to the rescue with Django Evolution. We’ve been using it since Review Board 1.0 alpha, and for the past several years, we’ve been maintaining it.

Here’s what it does, in a nutshell:

  • It lets us define what changes need to be made in a database-agnostic way, through “evolution files.”
  • It figures out what changes have already been applied (through a table that tracks those changes) and what changes still need to be applied.
  • It then converts those evolution files into a set of mutation operations, optimizes them, and then converts them to SQL statements suitable to the database.

Now, Django 1.7 finally has support for migrations, but of course the format is wildly different. For in-house projects, this is no big deal. You update the database one last time with Django Evolution, then use Django’s migrations from there on out. That doesn’t work with Review Board, though, since any user can go from any release to any release.

We cannot ever upgrade our Django dependency without solving this problem. Oh yeah, and Review Board extensions are affected as well.

So how do we handle this? Well, this is the game plan:

  • Write a converter to convert the evolution files to Django migration files.
  • Add a hook just before Django’s migration attempt, that imports the Django Evolution history into Django migration’s format. It would do this for any unimported apps, so we need to track those.
  • Write some compatibility shims that convert evolution commands to matching Django migration commands, and begin a deprecation phase.

Seems sorta straight-forward, right? Well, this stuff is always more complicated than it should be, so we’ll see.

I’m on step 1, and so far, this is looking like it’ll work. Expect some long posts with descriptions of how gracefully I’m bashing my head into the wall shortly.

Read More