RBCommons

A Sneak Peak Into the New RBCommons

Christian Hammond

Our team’s been hard at work bringing RBCommons to the cutting-edge Review Board 7 platform. This is a massive update with tons of new features, and we think it’s time to give you a glimpse of what’s coming:

Multi-commit review requests for Git and Mercurial

In the past decade, DVCS has become the norm, and many people work in local branches with lots of individual commits. Previously, any code changes posted to RBCommons would show up as a squashed diff, merely showing the difference between the upstream code and the tip of the local branch.

When using DVCS systems like Git or Mercurial, RBTools will now upload the commit history of your branch. You can look at the diff of the branch as a whole, but also dig down into each individual commit.

A list of commits for a review request, with summaries for each and selectors for choosing a range to view.

An improved review experience

Until now, your draft review requests, reviews, and replies were all managed separately, each with their own green draft banner. While it was obvious when there was a draft review request update or a draft review, sometimes draft replies could get lost in long threads.

The new unified banner always sits at the top of the page, and collects all your work together in one place. You can also use this to publish all your drafts at once with a single click and a single-email notification.

The new review banner, showing pending changes, one review, and one reply. A button saying "Publish All" to the right of that summary publishes all pending drafts. A "Review" drop-down to the right of the button controls the review. A "Describe your changes" field sits below it.

Dark mode

There’s nothing worse than staying up late to review code and feeling blinded by your screen. With dark mode, you can reduce eye strain and work comfortably no matter the time of day.

A typical review request, but shown in dark mode, with a dark cool-blue color palette.

And just so much more

There’s a lot more coming, but we wanted to keep this first message short. We’re working hard to make this transition as seamless as possible for you. Keep an eye out for more details about all the new features and a timeline for the rollout.

If you have any questions, please feel free to reply to this e-mail.

Thanks for using RBCommons! We’re excited to bring Review Board 7 to you.

Read More

ChangeLog: May 21, 2020 — Trial Limit Increases, New Releases, Student Wrap-Up

Christian Hammond

If you’re a regular follower of ChangeLog, you’ll notice we’ve gone from weekly to semi-monthly, and may be wondering what’s going on. Don’t worry, we’ll return to our regularly-scheduled ChangeLog in time.

We’ve been focusing heavily on wrapping up Review Board 4.0 development, testing things internally, and helping many of our support customers get out from under a backlog of internal support requests within their companies.

And just taking care of ourselves during a global pandemic.

So here’s some of what we’ve been busy with lately:

  • Increasing Power Pack and RBCommons trials
  • Several new releases (RBTools, kgb, and introducing babel-plugin-django-gettext)
  • Review Board 3.0.18 release preparation
  • Review Board 4.0 beta and RBTools 2.0 beta preparation
  • Wrapping up our semester with CANOSP students

Higher Power Pack/RBCommons Trial Lengths

We’ve increased the amount of time you have to give Power Pack or RBCommons a try. Now, when you download a Power Pack license, or sign up for a team on RBCommons, you have two full months to fully explore and use the products.

We’ve applied the new trial period to all existing RBCommons customers who are still in their trial.

If you’re a Power Pack user, and have a trial license, come talk to us for an extension.

New Releases

RBTools 1.0.3

Last month, we released RBTools 1.0.3, which was long overdue. We’re going to try to release RBTools releases more frequently going forward, and we have some good stuff prepared for 1.0.4 for Perforce users coming up soon.

We also have two new releases for some tools we use to help build Review Board: kgb, and introducing babel-plugin-django-gettext.

kgb 5.0

kgb is a Python module that helps with writing unit tests, adding support for function spies. This lets you spy on any function or method, whether in your own code or elsewhere, and track all calls made to the function and inspect the results of those calls.

It’s also used to override what happens when a function is called, mocking results or behavior. This goes far beyond the capabilities of Python’s own mock patching, and instead alters things at a bytecode level. Super useful when you want to fake results from urlopen, for example.

kgb 5.0 introduces support for:

  • Python 3.8
  • New spy assertion methods, providing detailed output when they fail
  • Support for spying on “slippery” functions (functions generated dynamically when referencing the function itself — common in some API-wrapping Python libraries, like Stripe)

babel-plugin-django-gettext 1.0

We use Babel to let us build modern JavaScript and export it to older browsers. Something Babel allows for is custom plugins to transform JavaScript, and we’ve introduced a new plugin to help us write better localized text.

babel-plugin-django-gettext lets us mark up strings using modern JavaScript tagged template literals (backtick strings) and convert them to use Django’s gettext localization methods.

When using the standard gettext support, lines are not allowed to wrap, meaning you end up with some very long lines of text to maintain, and if you want to include the contents of variables in the text, you have to wrap in this interpolate() call, which is a pain.

This plugin takes all the annoyance out of this. Instead of writing:

var s = interpolate(
    gettext('This is localizated text, and we can freely wrap lines how we want, or include variables like %(foo)s.'),
    {'foo': foo},
    true);

We get to write:

const s = _`
    This is localizated text, and we can freely wrap
    lines how we want, or include variables like ${foo}.
`;

Better, right?

If you use Babel and Django, give this plugin a try.

We’ll be releasing a new version soon with even better support for ngettext (used for strings that are based on singular/plural values) and combining with other tagged templates (like dedent).

Review Board 3.0.18 Release Prep

We’re getting close to a new Review Board 3.0.18 release. There’s a lot going into this one, but some highlights will include:

  • Preparation for GitHub and Bitbucket API/feature deprecations
  • Compatibility fixes for GitLab, Subversion, and Perforce
  • Improved API support for working with repositories
  • Faster SSH communication
  • Faster condensediffs for large MySQL databases
  • Lots of bug fixes

Expect 3.0.18 within the next two weeks.

Review Board 4.0 Release Prep

Work continues. We’ve had some people test 4.0 early, and found some regressions that pertain to extensions. We don’t want to release with those regressions in place, so we’re still iterating, but the good news is that the core product is looking pretty good now.

Remember, this release is a major architectural rewrite of the product, with equally major dependency updates, so there’s a lot to get right.

Meanwhile, we’re getting RBTools 2.0 ready for beta. This is meant to be used with Review Board 4.0, and features all the multi-commit review support, from posting changes to landing them. We’ll be shipping both at the same time.

CANOSP Student Wrap-Ups

We’ve talked before about the CANOSP student program we work with in Canada. Well, we’ve wrapped up our semester, and I can speak for the team when I say we’re going to miss working with this group.

By the way, if you’re looking to hire some strong developers coming out of college, we have plenty we can refer.

To wrap up their semester, they’ve put together some final demos of the work they’ve done, and we’d like to show them off.

Hannah Lin

Hannah worked this semester on a prototype for a new first-time setup guide for administrators, and some keyboard accessibility improvements in the diff viewer and modal dialogs, amongst other improvements. She’s also continuing on after the semester, working on a formatting toolbar for input fields.

Katherine Patenio

Katherine worked away on RBTools for most of the semester, fixing some bugs that shipped in RBTools 1.0.3, and completely reworking the rbt setup-repo experience (which we hope to ship in RBTools 2.0).

She also did a lot of work on investigating improvements to supporting users with different kinds of color-blindness, which she covers in this demo.

Monica Bui

Monica focused primarily this semester on keyboard navigation improvements in the New Review Request page (part of a big effort toward improved accessibility), and prototyping new guidance for filling in fields on a blank review request. We think that will pair nicely with work planned for Review Board 5.0.

Xiaohui Liu

Xiaohui worked on standardizing how we handle keyboard shortcuts, introducing a new registry on the page that anything can plug into to register shortcuts. This even offers a handy help screen, giving users an overview of all the keys can happily press to get their work done faster.

Xiaole Zeng

Xiaole’s projects covered help and accessibility improvements, such as adding a new Help menu to the top-right of every page (which could provide access to useful, relevant documentation), and making the review request infoboxes on the Dashboard less annoying and more keyboard-friendly. We’re looking to ship some of this in 4.0.

And that’s it for the moment

We’ll be back to a weekly format once we’ve gotten some of these releases wrapped up, and of course any time we have something pretty exciting to talk about.

In the meantime, if we can help with anything, reach out. You can also follow us on the community forum, Reddit, Twitter, Facebook, and YouTube if you want other ways to keep up with Review Board and Beanbag.

Read More

ChangeLog: April 2, 2020 — Catching Up

Christian Hammond 1 Comment

This is our first ChangeLog in several weeks. As you all know, the current pandemic has resulted in a lot of changes and hardships in the world. We’re doing fine here, and our team has stayed healthy and safe, if a little less productive than we’d like as we adjust and take care of our families.

David building a playground for the kids

Still, work never ceases, and it’s time to start keeping you all up-to-date again. Here’s a breakdown of what we’ll be covering today.

  • Support options for Review Board
  • Upcoming increases to RBCommons and Power Pack trial lengths
  • Upcoming releases of RBTools 1.0.3 and KGB 5.0.
  • Review Board 4.0 progress
  • New student demo videos

Getting Support for Review Board

More companies than ever are in full-on work-from-home mode, and this brings with it a lot of new work challenges that are, right now, often mixed with personal-life stresses.

We can help with at least some of that.

Our company offers support contracts for Review Board, which can be tailored to meet your company’s needs. We help with anything from basic Q&A and troubleshooting to custom builds and assistance with developing in-house integrations.

If you’re managing Review Board at your company, and are feeling a bit overwhelmed right now, please reach out. We can help. And we take support seriously.

Basic Support

Basic Support is pretty well-suited for smaller companies that need general troubleshooting, installation/upgrade assistance, or may have other questions.

We guarantee a response by the following day, but always aim for same-day (just depends on the support load).

Unlike our community support forum, all your support requests are handled privately on a dedicated support tracker, where you can manage tickets, provide confidential attachments, and more.

We don’t farm out our support to some outside party. We, the developers of Review Board, will handle your support. You’ll probably hear from me personally quite a bit.

Premium Support

This is a better option for the larger companies.

Or if you’re working on any in-house integrations or need priority bug fixes or need to use an older version of Review Board but may need some custom builds with fixes on occasion.

Or have some terrible emergency that needs to be resolved quick.

With Premium, there’s a same-day guarantee, 24/7/365. We’ll usually respond within an hour, especially if it’s an emergency. I will personally wake up and take care of your issue at 4AM if you need something.

So again, if things are crazy right now and you need a hand, contact us and we’ll talk options with you.

Increases to Trial Lengths

Since things are slower-moving right now (again, with the work-from-home status of so many companies), we want to make sure that you’re not in as much of a rush to evaluate either RBCommons and Power Pack.

So we’re going to be increasing the trial lengths of both from 30 days to 60.

This isn’t done yet, as we’re still preparing the codebases to change this over. In the meantime, if you’re a trial user of either, we’ll try to make sure to be proactive and increase your trial period manually.

If you’re already trialing either of this, contact us for a trial extension.

Upcoming Releases

RBTools 1.0.3

We’re finishing up this release now. It’s a big feature and bug fix release that we’ve been ironing out for a while.

The highlights are:

  • rbt land support for Mercurial
  • A much better commit editing experience (for rbt land and rbt patch)
  • Several bug fixes for various source code management systems and for Python 3 environments

We should have this release out next week.

KGB 5.0

KGB is our Python module for using function spies in unit tests. This lets you track when a function is called, with what arguments and results, and to even override what happens when that function is called.

It’s extremely powerful, and is a big part of how we maintain our large test suites.

We’re preparing a 5.0 release, which adds:

  • Python 3.8 support, with positional-only arguments
  • Workarounds for very corner-casey situations with method decorator that generate a new function every time it’s accessed (what we’re calling “slippery functions,” because they’re hard to hold on to)
  • Probably some new helpers for asserting the results of calls (TBD)

This should be released in the coming weeks. If you’re a Python user, I highly suggest giving KGB a try.

Review Board 4.0 Progress

Almost there.

We were going to get 4.0 in beta form by end of March. That was the goal. We hinted at this last time, and we were feeling good about it, but the impact from the pandemic changed some priorities.

So it’s delayed… I’m not going to give a new date at this point, but nearly everything is ready for beta. We just want to hammer on it some more first, make sure we’re pushing out a solid beta. Pretty much everything, including our extension ecosystem, is ready to go.

Student Demos

As you may know by now, we work with CS students every semester, mentoring them and helping them learn to contribute to real-world code bases through Review Board development.

They recently completed their second demo videos for the semester, showing off what they’ve built. Please take a look. I’m sure they’d love to hear some positive feedback on their videos:

Stay Safe, and Wash Your Hands

(Definitely the catch phrase of 2020, but it’s important!)

Again, if we can help with anything, reach out, or follow us on the community forum, Reddit, Twitter, Facebook, and YouTube if you want other ways to keep up with Review Board and Beanbag.

Read More

RBCommons: User Roles and Billing Updates

Christian Hammond 1 Comment

We’ve just gone live on a major update to the billing capabilities in RBCommons.

Try RBCommons without a credit card

With all the fraud and stolen credit card numbers out there, it’s no surprise that a lot of people wanted to try RBCommons to see if it was the right fit but weren’t comfortable providing their credit card information right away.

We’ve changed our trial so that you can sign up with only your name and e-mail address, and if you decide to keep using RBCommons, you can add your billing information later.

Separate administration and billing user roles

Many companies have a dedicated person for dealing with billing administration for services. Until now RBCommons has only had a single team administrator role, which provided access to both the billing information as well as everything else for the team. We’ve split up these responsibilities into new user roles:

  • A Technical Administrator can make changes to users, repositories, and other settings, but cannot change or see billing information.
  • A Billing Contact can see invoices and make changes to the billing information, as well as change which plan the team is on. They’ll also receive e-mails whenever we charge the attached card.
  • The Team Owner has access to all administration and billing capabilities. This is equivalent to the old team administrator role.

All team administrators have been updated to become Team Owners. To change a user’s roles, visit your Team Administration → Users page and click the pencil icon beside a user.

Improved invoices

Many countries require invoices to contain certain information, such as an official business name and address, or a tax ID. RBCommons now allows you to add this information in Team Admin → Account and Billing, and it will show up on your invoices.

If you’re a business located in the EU, you can put in your VAT ID and we’ll make sure that the generated invoices contain everything you need for your VAT filings.

If your country has invoice requirements that we haven’t met, please contact us.

Add billing e-mail recipients

You can now add additional e-mail addresses where you’d like any and all billing e-mails sent to. This is really useful if you have a purchasing department or some users who need to track receipts but don’t need access to RBCommons.

You can set these over in Team Admin → Account and Billing → Billing E-mails.

Update to the Privacy Policy

As part of this, we’ve made a small update to our Privacy Policy to list Quaderno as a third-party service used in our billing process. This is a good time to review your privacy choices under My Account → My Privacy Rights.

Feedback?

This has been in the works for a long time, and we’ll be iterating on it based on your feedback. So how’s it working for you? Let us know through the Need Help? button in the bottom-right of any page (opt in to Intercom in My Account → My Privacy Rights) or send us an e-mail at support@beanbaginc.com.

Read More

ChangeLog: December 19, 2019 — Wrapping Up for the Holidays

Christian Hammond

Hi everyone! Welcome to our final ChangeLog of the year.

We skipped last week, preparing for the big upcoming billing feature launch for RBCommons, which we discussed in November. That’s coming very soon, and once it’s done we’ll be getting back to our regular work on Review Board.

Since that’s the big project we’ve been working on, let’s talk about it some more. Buckle up. This is going to be a long one.

Software Companies, Credit Cards, and Tax Requirements

We have four main goals for our RBCommons billing update:

  1. Give teams better invoices and more control over who can manage billing
  2. Better support credit card safety measures like Strong Customer Authentication and 3D Secure
  3. Be able to better meet invoicing and tax standards in more countries
  4. Making RBCommons team sign-up easier for everyone

We talked about the first one before. So let’s focus on 2 and 3. We’ll be discussing these in simplistic terms — the reality is more complex.

Strong Customer Authentication

Strong Customer Authentication, or SCA for short, is a regulation in the EU designed to reduce credit card fraud. It went into effect in September 14, 2019, and effectively adds a step to credit card charges where the purchaser must authenticate/verify the charge, typically using a verification method called 3D Secure.

This impacts you if you do any business with the EU.

Now, there are exemptions to this. Recurring charges may be exempt. Charges from the US or other countries may be exempt (likely temporarily — the world just isn’t ready to fully comply yet). Other transactions considered low-risk are also likely to be exempt. This all depends, though, on the credit card company and the reputation of the seller.

Enter Stripe, Our Billing Provider

We use Stripe, which takes care of most of this. It’s pretty great. However, we weren’t quite in shape to really leverage their support, for a couple reasons:

  1. We were sending our own receipt and failed charge e-mails to customers, and they weren’t accounting for any SCA-related requirements
  2. While our e-mails went out to all team administrators, Stripe will only send their own e-mails to a single e-mail address, which may not even be current (if the team has been around a while and people have moved on), meaning they may not ever get a chance to verify charges

We knew we wanted to rip out all our e-mails, but Stripe’s one-address limitation was causing us headaches.

Now technically Stripe can send to multiple e-mail addresses, but this can only be configured manually through their Stripe Dashboard UI. The API doesn’t support this yet. So we were stuck.

Enter MailGun Routes

We use MailGun as our e-mail provider, and it has a handful of really nice features. One of them, Routes, allows for setting up rules to match incoming e-mails and do something with them, such as forwarding them on to other addresses or to a WebHook.

We found that we can dynamically create routes that match an incoming e-mail address unique to the team and forward it along to all team users responsible for billing. We can then assign that unique e-mail address to Stripe. They look something like this:

match_recipient('.*@mydomain')
forward('user1@example.com, user2@example.com, user3@example.com')

We can create these when new teams are created, update them whenever the list of billing contacts change, and delete them when the team is deleted. Problem solved!

(But seriously, Stripe, add multiple e-mail address support to your API.)

Invoices and Taxes

So this is the big challenge. Complying with international taxes is hard. There just isn’t really a lot of infrastructure out there to help deal with this, and every country has different requirements. I’m not going to give any advice here, but I’m going to point you all to some useful things we’ve found.

Enter Quaderno, The Tax Guide

First, Quaderno. This service provides a number of tools for helping with tax compliance:

  • You can use it for all invoices and charges if you like, or pair it with something like Stripe
  • It can let you know if you’re missing any customer information necessary to validate them for tax purposes
  • It will show you a breakdown of what countries you currently owe taxes to, and give you the necessary information needed to file those taxes
  • It can even provide a sort of store front, if your needs aren’t too complex

We found Quaderno helpful not just for the tools it provides, but the information. They have an extensive knowledge base on how to comply with tax laws in multiple countries, including tax requirements, invoicing requirements, tax rates and categories, and how to apply for a tax ID in the country.

Just search for “Quaderno <region>” and you’ll find a wealth of information. For instance, here’s their Guide to EU VAT.

Collect Billing and Tax Details

A lot of services aim to collect as few details as possible from customers. This seems like a good approach, especially when you think of the privacy enhancement bills like the GDPR and the California Consumer Privacy Act, but you’ll need to collect a bit more to be tax-compliant:

  1. A full and proper billing address for the company, including country
  2. A tax ID for the business that matches that country

You’ll need to validate these to make sure you’re not accidentally enabling fraud. Most countries make this your problem. Quaderno and other services can help with this.

This information also needs to be on the invoices, and in fact you may need additional details including your own tax ID in that country. Both Stripe and Quaderno can help with this (Quaderno is better geared toward flexible, compliant invoices, but Stripe is better integrated with the rest of the billing process).

So a big part of what we’ve been putting together includes:

  • New settings for collecting company addresses and tax IDs, and validating them
  • Switching to Stripe e-mails and invoices for better compliance
  • Connecting our stuff up with Quaderno to better track tax requirements
  • Augmenting Stripe invoices with information required to meet the requirements in some countries, based on the customer’s billing location
  • In-house processes for managing all this complexity

One More Thing: Credit Card-Free RBCommons Trials

RBCommons currently requires a credit card to sign up for the trial. We’ve had it this way for a long time, since it’s easier to seamlessly turn a trial into a paid plan without interruption, and a lot of our early customers were already familiar with us and had trust in our service. As we’ve grown, though, our customer base has widened, and we’ve wanted to remove this step to help make it as easy as possible to get started.

As part of our big billing update, we’re removing the credit card requirement during setup, and instead guiding people to provide it before their trial expires. We hope this will make more people feel comfortable giving RBCommons a try, and experiencing the type of code and document review we offer.

This will all be launching Very Soon Now (TM).

That’s It for 2019!

This is our last ChangeLog for the year, but we’ll be back early 2020. Keep following us on our blog, Reddit, Twitter, Facebook, and YouTube.

We hope everyone has a wonderful and relaxing holiday season! We’ll be taking some time off to spend with our loved ones (but don’t worry support contract customers — David and I are still on call if you need us).

Read More

ChangeLog: November 7, 2019 — Multi-Repo Diffs, Admin UI, and RBCommons

Christian Hammond

This week in ChangeLog, let’s talk about three projects we’ve been working on this week:

  • Multi-repository diffs
  • The latest administration UI improvements
  • Billing improvements and user roles in RBCommons

Multi-Repository Diffs

It’s pretty common to work with a lot of repositories at the same time, linking them together using Git submodules, SVN externals, or some other method. Sometimes you need to make changes across many repositories at once, and it’d be helpful to post those changes as one review request.

The problem is that diffs made across repositories aren’t always so useful. Not only do they generally lack repository information for each file, but Review Board itself assumes a single repository per review request. This matters because in order to show the diff viewer, Review Board needs to load each source file from the repository before it can apply the diff.

We’re trying to work toward multi-repository diff support as part of Review Board 4.0. Our existing DVCS work got us part-way there. The remaining pieces are/were:

  1. Updating our diff parsers to allow additional information (such as repository details) to be extracted and stored
  2. Updated our FileDiff model (representing a parsed file from a diff) to link to a Repository, and updating code to use that instead of ReviewRequest.repository
  3. Getting this information into diffs

Step 1 is done as of this week, and step 2 is mostly done. Step 3 is where things get interesting.

We talked last week about DiffX, our initiative to make diffs better. We’re hoping to start using DiffX from within RBTools and to inject repository information into the diff.

This is a longer-term goal, though. We’re exploring some options shorter-term, and talking to one vendor about natively providing this information in diffs they generate.

So will this be usable in Review Board 4.0? Kind of. We should have the core functionality all done, but not much may take advantage of this at first. Long-term, we’ll introduce multi-repository diff support for more types of repositories, and it’ll be amazing.

Administration UI Updates

Another week, another batch of screenshots to share of the new administration UI. We’ve completed the main database page and the Change List page, which lists all changes made to the database for a specific type of model (table).

These build upon new CSS components coming to 4.0, which offer slide-out action drawers and filters for datagrids, inline warning/error/info alerts, and more goodies that extension authors can use for their own projects.

Next up are the Settings and Change Form pages, which allow for making changes to models in the database.

Once these are done, we get to upgrade Django!

RBCommons Roles and Billing Updates

RBCommons, our Review Board SaaS, has received a lot of our attention lately. Particularly in the areas of billing, team administration, and sign-up.

Much of this work is based on customer feedback. As RBCommons has grown, our user base has moved from primarily Silicon Valley startups to an international community of companies, organizations, and educational institutions of all sizes. And some of those need a bit more from us when it comes to how they manage their teams and handle billing-related matters.

As part of our plans, we’re working on:

  • Splitting the existing Team Administrator role into three new roles:
    1. Team Owner (capable of making all changes to an account, including cancelling and changing plans)
    2. Team Administrator (capable of managing users, repositories, review groups, etc., but cannot manage anything billing-related)
    3. Billing Contact (can view invoices, change payment information, and receive billing-related e-mails, but nothing else)
  • Improving our invoices, showing more useful information to help users better meet any company-internal, regulatory, or tax requirements they might have
  • Adapting our billing process to work better with credit card billing changes that are or will be going into effect in some countries, to ensure these customers’ payments go through without any headaches
  • Improving the sign-up flow, to help get teams up-and-running a bit faster

We’ll talk about all this in more detail later when we launch our next big update to RBCommons.

Next Week

We’ll have more Administration UI work to show off, and with it a new collection of CSS Components to talk about. We’ll also go over what our CANOSP students from the University of Alberta have been up to.

If you have any questions, or anything you’re curious about and want us to cover, please reach out on our community forum.

We’re also on reddit /r/reviewboard, Twitter, and Facebook, if you want other ways to keep up with the latest.

Read More

ChangeLog: October 24, 2019 – The Python 3 and Django 1.11 Migration

Christian Hammond 2 Comments

Hi, Christian Hammond here. Welcome back to ChangeLog, where we cover the latest going on in Review Board, RBCommons, and Beanbag in general. It’s been a while since we’ve posted one of these, but we’re aiming to bring this back as a weekly series.

There’s a lot to talk about.

The Short Version

  • We delayed Review Board 4.0 to bring Django 1.11 and Python 3 support. This will not be released by Python 2’s End Of Life date.
  • Python 3 and Django 1.11 support has been in the works for a long time, but is far more complex than it may seem.

The Longer Version: Where are we today

We’ve had four main focuses this year:

  • Getting Review Board 4.0 beta 1 ready to launch. This is way behind our original schedule, but intentionally so — we’ll go over the reasons why in a minute.
  • Bringing integrations options to RBCommons and improving team management, billing, and signup.
  • Building out new functionality for Power Pack (PDF diffing, and in-progress authentication improvements).
  • Growing our business (the prior two tasks were part of that), supporting customers, and assisting some with large projects of their own.

This along with a multitude of smaller tasks taking our time throughout the year (and, on a personal note, dealing with the aftermath of the Paradise, CA “Camp Fire.”)

Most of you are here to find out about Review Board, so let’s dive into that.

The “World Update”: Review Board 4.0, Python 3, and Django 1.11

Our original goal for Review Board 4.0 was to bring support for multi-commit review requests and to ship that. This was a large project, as Review Board was originally written before DVCS was common (Subversion and Perforce were the primarily open source and enterprise solutions).

The problem though, is Python 2 End Of Life is coming up fast, and if we kept on schedule with Review Board 4.0, it would be a long time until we’d support Python 3. So we decided to delay 4.0 in order to get it ready for Python 3.

Note that we also still need to support Python 2.7, since many companies are still heavily tied to 2.7 (older distros, custom extensions and scripts).

So what’s hard about supporting Python 3? Welllll….

The Python Compatibility Problem

Python 3 is leaps and bounds better than Python 2 in most ways, but porting a large and complex product from Python 2 to 3 is even harder than you think.

Most Python users know of the major differences:

  • Modules have moved and functions have been renamed
  • The default string type has changed from byte strings to Unicode
  • Some operators have changed
  • There’s new syntax additions
  • etc.

We thought we prepared years ago to make this move easy. We used Unicode strings in every file. We used the six module to help with using the right modules, types, and functions.

In the end, it was harder than we expected. Our biggest challenge was definitely the Byte Strings vs. Unicode Strings differences. We thought we were in good shape for this, but we weren’t close.

Review Board does a lot of text processing. We’re parsing uploaded diffs, pulling source files from repositories, matching those up and applying the patch to the source, and generating side-by-side diffs. Much of this logic is over 10 years old, even so, we put a substantial amount of work in preparing for Python 3 years ago, so we were shocked by how much we got this wrong.

The problem comes down to the differences in how Python 2 and 3 would handle shoving two different string types together, which is very easy to do accidentally. Python 2 would roll with it, as in many cases the string types were compatible (and would automatically encode/decode so long as the content was basically ASCII). In Python 3, they outright break — which is a better approach, but hard to transition to — and it led us down a rabbit hole of problems.

Missing b'...' prefixes, changes in string return types from Python functions, functions that are no longer compatible with both string types, and very difficult decisions to make regarding compatibility with third-party SCMs.

All the little regressions and inconsistencies added up. Here’s a few more examples:

  • Using Python’s pickle library was a mess. Defaults have changed and pickled data became incompatible (due to module reference and string type changes). We had to build in compatibility layers here and test them thoroughly.
  • Anything that even subtly/unintentionally relied on dictionary sort orders would break.
  • Getting data in/out of processes, streams, and many other objects and functions is now way more sensitive to encoding issues, and required a lot of careful rewrites and testing.
  • Many functions (in Python and third-party modules) that used to return lists now return generators, due to their reliance on other functions that changed return types, and this can cause all sorts of subtle behavioral changes.

We’ve spent a lot of time tracking down issues that might immediately crash or might affect data several stages down, and sometimes be traced to something outside our codebase.

Fixing some Python issues means upgrading third-party modules, which can introduce their own new set of changes, regressions, and new rounds of work.

And the biggest source of that was Django.

The Django Problem

Django 1.6 (the version we’ve been using) breaks on Python 3.6+. This meant we had to upgrade to a newer version, something we’ve been putting off.

Django 1.7 introduced built-in support for database migrations. Quite nice for many projects, but the design was sub-optimal for applications like ours (upgrades could take hours or days longer than our approach) and was fundamentally incompatible with our own Django Evolution.

Many other core components of Django (the foundation for the administration UI, template rendering, URL management, forms, and all sorts of other common and obscure parts) have also largely changed over time in some pretty important ways.

We needed a solution for all of this, and we knew that would take time. To compensate we had to:

  1. Rewrite Django Evolution completely to coordinate evolutions and migrations and support modern Django: ~2 years of work, off and on, with hard problems to ponder
  2. Rewrite our administration UI completely to disconnect from Django’s (ensuring we don’t break again when we move to Django 2.x): estimated ~3 months of steady work, still in progress
  3. Build compatibility layers to help keep older code working and help with new code: ~2 months
  4. Just porting in general, updating dependencies, etc: ~3 months

We’re not done, but we’re close.

The Extensions Problem

Review Board is built to allow third-parties to extend its functionality, modify behavior, and link up with other in-house services. We try to be careful not to break extension functionality, and over time we’ve gotten more strict about providing compatibility and an upgrade path for functionality we want to deprecate or change.

The Python 3 and Django 1.11 upgrade is going to affect just about everybody who’s writing an extension. Some of the work we’ve done on adding compatibility layers will help with this process, but it’s going to mean a longer upgrade cycle for some companies.

We knew from the beginning that we’re going to have our hands full helping these companies out (as part of our support contracts), and that it’d be in our best interest to delay the release and break everything all at once instead of spreading out the breakages across multiple releases, and to also give ourselves time to figure out how best to minimize those breakages.

Most projects moving to Python 3 or to newer versions of Django don’t have to worry about this domino effect in the same way.

Timetable?

I hesitate to say when beta 1 will be done (been wrong before), but I can say that the last major piece of development is wrapping up. We’ll be kicking the tires on it, and want to get a beta out as soon as it’s stable enough.

We may not enable Python 3 builds for beta 1, focusing instead on Django 1.11 testing (Python 3 support is still going to be in development during this time), however we’re working with select customers on real-world testing against Python 3.

Community Questions

Every week, we’d like to address some questions, concerns, suggestions, etc. from the community. If you have any questions for next week, please reach out to us.

Q. The community forum seems quiet at times. Why is that?

A. Support requests from companies with support contracts are conducted over a separate support tracker. We always prioritize these support requests over any other work, and increasingly more companies are moving to this method of support for Review Board.

Q. Will Review Board 4.0 ship with Python 3 before the Python 2 End Of Life date?

A. No, it’s going to miss that date. Python 2.7 is still going to be required for now.

Q. There’s years of open review requests on reviews.reviewboard.org. Why is that?

A. We work with university students every semester to help prepare them for their jobs in the tech industry. They spend their semester building features for Review Board, most of which are prototypes. These make up the majority of the review requests on there. Others are contributions that might be outdated, might have been missed, might be incomplete, or might just not have been reviewed yet.

Next Week

We’ll be going over our new CSS component standard for the project and dive deeper into the new administration UI.

Again, if you have any questions, or anything you’re curious about and want us to cover, please reach out on our community forum.

Read More

Connect RBCommons to Slack, Travis CI, Asana, and more

Christian Hammond

All RBCommons plans now support integrating with third-party chat, continuous integration, and project management services, with more services to come.

You can connect with Slack or Mattermost and keep your team members notified whenever a review request is created or updated, or when any new discussions take place.

Slack Integration

Automatically trigger builds of uploaded changes on Travis CI or CircleCI, ensuring product builds still pass. You can even run automated code review tools and report status using RBTools.

Link your Asana or Trello tasks with your review requests, and add all your code review work to your status report automatically using I Done This.

Asana Integration

Each integration configuration can specify the rules under which the integration will run. For instance, when a build should take place or which Slack teams/channels should be used for which repositories or review groups.

Configuring Condition Rules

You can define as many integration configurations as you like, up to your plan’s limit:

  • Starter: 1
  • Medium: 10
  • Large: 25
  • Enterprise: 50

See our integration guides to get started today.

Read More

RBTools 1.0 is here!

Christian Hammond

RBTools has been an important part of the life of Review Board users for many years. While it started off as a single tool for posting review requests, its feature set has evolved with time, turning into an extensible set of tools and APIs for talking to Review Board.

Today, we’re finally pulling RBTools out of the 0.x era with the release of RBTools 1.0.

Compatible with Python 3

Both the RBTools commands and the Python API now support Python 2.7 and 3.5+.

(Please let us know if you hit any issues on Python 3, as this is still pretty new.)

Better Repository Detection and Git Support

RBTools now does a better job determining which repository it’s working with, in case there’s confusion. For example, a Mercurial repository nested in a Git-managed home directory will no longer cause problems.

Git repositories in particular are now easier to work with. When generating a diff, RBTools now looks for the nearest upstream parent commit or branch, instead of requiring that users or repositories configure a specific tracking branch.

Publish Automated Reviews

Writing your own automated review solutions for Review Board 3.0 or RBCommons just became easier through the new rbt status-update command. Your scripts can use it to file a pending status update on a review request (showing that checks are being performed) and then update it to say that all is well or to report issues that need to be fixed.

This is useful for in-house continuous integration setups where you’re analyzing code for errors, style issues, documentation, or any other requirements you might have.

Easily Land Complex Dependent Changes

rbt land can now land multiple review requests tied together using the Depends On field.

This works with -r to take the ID of the review request you want to land. It will figure out which review requests must land before it and in which order. For example, if review request 3 depends on 2, which depends on 1, you can run:

$ rbt land --recursive -r 3

Instead of:

$ rbt land -r 1
$ rbt land -r 2
$ rbt land -r 3

This is a precursor to the new DVCS support coming soon in Review Board 4.0.

And That’s Not All

  • rbt setup-completion was added to enable auto-completion of RBTools commands and arguments in Bash and ZSH shells.

  • rbt alias was added to help you list and test out your custom aliases.

  • rbt post –submit-as can now automate posting review request updates, and not just new review requests, on a user’s behalf.

  • rbt post -m and rbt publish -m let you specify a custom description of your draft’s changes when publishing (equivalent to filling out the “Describe your changes” box when publishing in the browser).

  • rbt post –trivial-publish and rbt publish –trivial let you publish trivial updates to a review request without sending out e-mails to everyone (when using Review Board 3.0 or RBCommons).

  • rbt status now lists the review state and local branch for each review request you have up for review.

  • Warnings and errors in command output is now specially highlighted to help it stand out.

  • Several fixes and improvements for Git and Subversion compatibility.

  • The API has been improved, supporting extra_data fields and easier pagination of resources.

And plenty of other fixes and improvements. See the release notes for the full list of changes in 1.0.

Download It Today!

RBTools is out today for Windows, Linux, and Mac. Head on over to the downloads page for installation instructions.

Read More

A New Era of Privacy – Beanbag, Review Board, and You

Christian Hammond

If you’ve been on the Internet at all in the past few weeks, you’ve likely been bombarded by e-mails from every service you’ve ever used telling you that they’ve updated their privacy policy.

Yes, we’ve updated ours too, but we want to talk to you about how we manage your data, the greater picture of privacy on the Internet, the European Union’s new GDPR legislation, and why this all matters to you, no matter where you are in the world.

GDPR: A new standard for privacy

On May 25th, the GDPR (General Data Protection Regulation) goes into effect in the EU. It gives users there an unprecedented level of control and insight into their personal information. Amongst other things:

  • People must be able to change, delete, or request copies of their personally identifiable information
  • Companies need a valid legal basis for the usage of that information (which may require getting consent for that use)
  • Handing that data off to third-parties also requires a legal basis, and must be documented in a privacy policy
  • That privacy policy must be able to be read and understood without a law degree

You must admit, that’s pretty nice. Though companies are not required to give these rights to non-EU residents, many (including us) are treating this as a new global standard.

Some parts of the GDPR are a bit vague and not all companies see eye-to-eye on the level of control you should have. We’re hoping our approach goes above and beyond.

Our new privacy guarantees

We’ve always collected as little data as needed. We don’t need much, except to provide services to you, to aid in team communication, and to make use of third-party services we trust who help us run our business and provide support to you.

Still, under the GDPR, there was more for us to do. So here’s what we’re promising:

  • We’ll continue to only collect what we strictly need, and to document it clearly in our Privacy Policy.
  • We’ll continue to give you control of your data, and handle deletion and alteration requests, as we always have.
  • We’ve updated our services to request your consent (and give you full control over it at any time) for any optional usage of your personal information, and any usage we strictly require to run our services effectively will be clearly documented.
  • We’ve never sold your information and never will.
  • We’re extending the rights granted by the GDPR to all users of our products, everywhere.
  • If you ever have any questions or concerns about your data, we’re always here to help.

To help, we’ve built a whole new privacy-focused framework in Djblets to help with privacy guarantees and consent requests. All our software will be using this and we’ll be encouraging Review Board extension authors to use it. We’ll talk about this in more detail in an upcoming post.

What to expect by May 25th

Our Privacy Policy is up now, and will take effect on May 25th, 2018.

We’ll be activating the enhanced privacy support on RBCommons, reviewboard.org, reviews.reviewboard.org, and Splat in time for the 25th. If you’re a user on these, the next time you connect you’ll be asked to accept the Privacy Policy and to allow or block usage of your information for some services.

We’ll also be releasing Review Board 3.0.7 and Djblets 1.0.6, which are privacy-enhanced, optionally allowing for Terms of Service and Privacy Policy URLs and GDPR-compliant consent functionality. Many servers may not need this, but it’ll be available for those that do.

If you want to change, delete, or request any of your personal information from our servers, or want more information on all this, reach out to us at any time and we’ll help. You don’t need to wait for May 25th.

Read More