ChangeLog: January 23, 2020 — Student Sprint and Django Upgrade

What a week it’s been. We have two topics to go over: Last weekend’s student mentorship sprint, and a status update on the big Django upgrade.

The student sprint was great!

Last weekend was our student sprint for CANOSP, kindly hosted by the Startup Edmonton in Edmonton, Canada. There, we met up with our new students from the University of Alberta and University of British Columbia, and spent the weekend getting to know them, teaching them about Review Board and how we do things here, and letting them loose on projects.

Students working hard on projects

David giving an architectural overview

David giving an architectural overview (close up)

We may have been sleepy most of the time (these start early!), and VERY cold (had to sit by a stack of ice cubes to keep warm), but everyone had a lot of fun, and have since been hard at work on their projects.

The main focuses for this semester are accessibility and usability. We’re working to make the product easier to use, improving keyboard navigation, and experimenting with ways to offer useful inline help. Much of this work is slated for Review Board 4.0.

Django upgrade time!

This week also marks the end of Django 1.6 in Review Board. We’ve been working for a long time on getting onto a newer version of Django, which has been a much more complex project for us than for most, and we’re finally ready to bump our Django requirement to 1.11.

Diff showing the change to upgrade to Django 1.11

For those paying attention, Django itself is at 3.0.2, but 1.11 is the last version to support Python 2.7. While Python 2.x is now end-of-life’d, that doesn’t mean it’s not in active use in enterprise, and frankly, many of our users are just not ready to upgrade. So Review Board 4.0 will continue to be providing compatibility for 2.7.

By the weekend, we should be on 1.11, and then we’ll be getting ready to test this in production. If all goes well, a beta will follow soon.

If you build extensions for Review Board, you’re going to need to make some changes to support Django 1.11. We have a bunch of useful information on Django updates on our wiki, which we’ll also include in the release notes. Make sure you give the beta a try and begin porting early.

We offer support contracts that cover development assistance, if you need it.

Back to work!

If you have any questions, or anything you’re curious about and want us to cover, please reach out on our community forum.

We’re also on Reddit (/r/reviewboard), Twitter, Facebook, and YouTube (featuring student project videos!) if you want other ways to keep up with what we’re up to.

Read More

ChangeLog: January 16, 2020 — Review Board Status and New Students

Hi everyone. Welcome to the first ChangeLog of 2020! We skipped last week due to the new Review Board 3.0.16 release, so let’s dive in and get caught up.

Review Board 3.0.16

Last Tuesday, we released Review Board 3.0.16. It’s been a long time since 3.0.15, and there are two reasons for this:

  1. Most of our attention of late has been on completing the remaining architectural work on Review Board 4.0 (Python 3 and Django 1.11 porting) and RBCommons user roles and billing updates.
  2. We’ve been trying to carefully design and implement some large backend improvements for repositories and repository configuration, in collaboration with another vendor, and wanted to get it right before we released.

We’ve discussed the repository improvements before, so read that if you want to learn a bit more, but the gist is that we’re giving SCMTools (repository backends) a lot more flexibility in how they present repository configuration, how they’re registered for use in Review Board, and how extra data for repositories get stored. This will lead to some significant improvements in the coming months for a couple of our supported repository types.

Now unless we find some major bug fixes in 3.0.16, it’ll probably be a little while until 3.0.17. We have a backlog of RBTools work we plan to release, and we’re still trying to get 4.0’s architectural rework done.

Review Board 4.0 Status

Most of the rewritten administration UI is in a usable state, and we’re just getting it all ready to be reviewed and landed. After this, we’ll be bumping our Django dependency and performing a bunch of real-world usage tests, just to make sure there isn’t some big breakage some place.

(If you want to learn more about the administration UI work and how it relates to Django and the release, read the ChangeLogs from October 24, 2019, October 31, 2019, and November 7, 2019.)

Once we’re happy, we’ll ship 4.0 Beta 1. Almost there!

New Semester, New Students

Every semester, we take on a batch of CS students eager to work on some real-world project, currently as part of the CANOSP program run by the University of Alberta, Canada.

It all starts this weekend at a get-together in Edmonton, Canada, where we’ll be helping five new students get set up, go over architecture and standards, and start them on their projects.

We’ll talk more about what they’ll be working on next week, but they mostly center around quality-of-life improvements to Review Board.

Wrapping Up The Week

If you have any questions, or anything you’re curious about and want us to cover, please reach out on our community forum.

We’re also on Reddit (/r/reviewboard), Twitter, Facebook, and YouTube (featuring student project videos!) if you want other ways to keep up with what we’re up to.

Read More

RBCommons: User Roles and Billing Updates

We’ve just gone live on a major update to the billing capabilities in RBCommons.

Try RBCommons without a credit card

With all the fraud and stolen credit card numbers out there, it’s no surprise that a lot of people wanted to try RBCommons to see if it was the right fit but weren’t comfortable providing their credit card information right away.

We’ve changed our trial so that you can sign up with only your name and e-mail address, and if you decide to keep using RBCommons, you can add your billing information later.

Separate administration and billing user roles

Many companies have a dedicated person for dealing with billing administration for services. Until now RBCommons has only had a single team administrator role, which provided access to both the billing information as well as everything else for the team. We’ve split up these responsibilities into new user roles:

  • A Technical Administrator can make changes to users, repositories, and other settings, but cannot change or see billing information.
  • A Billing Contact can see invoices and make changes to the billing information, as well as change which plan the team is on. They’ll also receive e-mails whenever we charge the attached card.
  • The Team Owner has access to all administration and billing capabilities. This is equivalent to the old team administrator role.

All team administrators have been updated to become Team Owners. To change a user’s roles, visit your Team Administration → Users page and click the pencil icon beside a user.

Improved invoices

Many countries require invoices to contain certain information, such as an official business name and address, or a tax ID. RBCommons now allows you to add this information in Team Admin → Account and Billing, and it will show up on your invoices.

If you’re a business located in the EU, you can put in your VAT ID and we’ll make sure that the generated invoices contain everything you need for your VAT filings.

If your country has invoice requirements that we haven’t met, please contact us.

Add billing e-mail recipients

You can now add additional e-mail addresses where you’d like any and all billing e-mails sent to. This is really useful if you have a purchasing department or some users who need to track receipts but don’t need access to RBCommons.

You can set these over in Team Admin → Account and Billing → Billing E-mails.

Update to the Privacy Policy

As part of this, we’ve made a small update to our Privacy Policy to list Quaderno as a third-party service used in our billing process. This is a good time to review your privacy choices under My Account → My Privacy Rights.

Feedback?

This has been in the works for a long time, and we’ll be iterating on it based on your feedback. So how’s it working for you? Let us know through the Need Help? button in the bottom-right of any page (opt in to Intercom in My Account → My Privacy Rights) or send us an e-mail at support@beanbaginc.com.

Read More

ChangeLog: December 19, 2019 — Wrapping Up for the Holidays

Hi everyone! Welcome to our final ChangeLog of the year.

We skipped last week, preparing for the big upcoming billing feature launch for RBCommons, which we discussed in November. That’s coming very soon, and once it’s done we’ll be getting back to our regular work on Review Board.

Since that’s the big project we’ve been working on, let’s talk about it some more. Buckle up. This is going to be a long one.

Software Companies, Credit Cards, and Tax Requirements

We have four main goals for our RBCommons billing update:

  1. Give teams better invoices and more control over who can manage billing
  2. Better support credit card safety measures like Strong Customer Authentication and 3D Secure
  3. Be able to better meet invoicing and tax standards in more countries
  4. Making RBCommons team sign-up easier for everyone

We talked about the first one before. So let’s focus on 2 and 3. We’ll be discussing these in simplistic terms — the reality is more complex.

Strong Customer Authentication

Strong Customer Authentication, or SCA for short, is a regulation in the EU designed to reduce credit card fraud. It went into effect in September 14, 2019, and effectively adds a step to credit card charges where the purchaser must authenticate/verify the charge, typically using a verification method called 3D Secure.

This impacts you if you do any business with the EU.

Now, there are exemptions to this. Recurring charges may be exempt. Charges from the US or other countries may be exempt (likely temporarily — the world just isn’t ready to fully comply yet). Other transactions considered low-risk are also likely to be exempt. This all depends, though, on the credit card company and the reputation of the seller.

Enter Stripe, Our Billing Provider

We use Stripe, which takes care of most of this. It’s pretty great. However, we weren’t quite in shape to really leverage their support, for a couple reasons:

  1. We were sending our own receipt and failed charge e-mails to customers, and they weren’t accounting for any SCA-related requirements
  2. While our e-mails went out to all team administrators, Stripe will only send their own e-mails to a single e-mail address, which may not even be current (if the team has been around a while and people have moved on), meaning they may not ever get a chance to verify charges

We knew we wanted to rip out all our e-mails, but Stripe’s one-address limitation was causing us headaches.

Now technically Stripe can send to multiple e-mail addresses, but this can only be configured manually through their Stripe Dashboard UI. The API doesn’t support this yet. So we were stuck.

Enter MailGun Routes

We use MailGun as our e-mail provider, and it has a handful of really nice features. One of them, Routes, allows for setting up rules to match incoming e-mails and do something with them, such as forwarding them on to other addresses or to a WebHook.

We found that we can dynamically create routes that match an incoming e-mail address unique to the team and forward it along to all team users responsible for billing. We can then assign that unique e-mail address to Stripe. They look something like this:

match_recipient('.*@mydomain')
forward('user1@example.com, user2@example.com, user3@example.com')

We can create these when new teams are created, update them whenever the list of billing contacts change, and delete them when the team is deleted. Problem solved!

(But seriously, Stripe, add multiple e-mail address support to your API.)

Invoices and Taxes

So this is the big challenge. Complying with international taxes is hard. There just isn’t really a lot of infrastructure out there to help deal with this, and every country has different requirements. I’m not going to give any advice here, but I’m going to point you all to some useful things we’ve found.

Enter Quaderno, The Tax Guide

First, Quaderno. This service provides a number of tools for helping with tax compliance:

  • You can use it for all invoices and charges if you like, or pair it with something like Stripe
  • It can let you know if you’re missing any customer information necessary to validate them for tax purposes
  • It will show you a breakdown of what countries you currently owe taxes to, and give you the necessary information needed to file those taxes
  • It can even provide a sort of store front, if your needs aren’t too complex

We found Quaderno helpful not just for the tools it provides, but the information. They have an extensive knowledge base on how to comply with tax laws in multiple countries, including tax requirements, invoicing requirements, tax rates and categories, and how to apply for a tax ID in the country.

Just search for “Quaderno <region>” and you’ll find a wealth of information. For instance, here’s their Guide to EU VAT.

Collect Billing and Tax Details

A lot of services aim to collect as few details as possible from customers. This seems like a good approach, especially when you think of the privacy enhancement bills like the GDPR and the California Consumer Privacy Act, but you’ll need to collect a bit more to be tax-compliant:

  1. A full and proper billing address for the company, including country
  2. A tax ID for the business that matches that country

You’ll need to validate these to make sure you’re not accidentally enabling fraud. Most countries make this your problem. Quaderno and other services can help with this.

This information also needs to be on the invoices, and in fact you may need additional details including your own tax ID in that country. Both Stripe and Quaderno can help with this (Quaderno is better geared toward flexible, compliant invoices, but Stripe is better integrated with the rest of the billing process).

So a big part of what we’ve been putting together includes:

  • New settings for collecting company addresses and tax IDs, and validating them
  • Switching to Stripe e-mails and invoices for better compliance
  • Connecting our stuff up with Quaderno to better track tax requirements
  • Augmenting Stripe invoices with information required to meet the requirements in some countries, based on the customer’s billing location
  • In-house processes for managing all this complexity

One More Thing: Credit Card-Free RBCommons Trials

RBCommons currently requires a credit card to sign up for the trial. We’ve had it this way for a long time, since it’s easier to seamlessly turn a trial into a paid plan without interruption, and a lot of our early customers were already familiar with us and had trust in our service. As we’ve grown, though, our customer base has widened, and we’ve wanted to remove this step to help make it as easy as possible to get started.

As part of our big billing update, we’re removing the credit card requirement during setup, and instead guiding people to provide it before their trial expires. We hope this will make more people feel comfortable giving RBCommons a try, and experiencing the type of code and document review we offer.

This will all be launching Very Soon Now (TM).

That’s It for 2019!

This is our last ChangeLog for the year, but we’ll be back early 2020. Keep following us on our blog, Reddit, Twitter, Facebook, and YouTube.

We hope everyone has a wonderful and relaxing holiday season! We’ll be taking some time off to spend with our loved ones (but don’t worry support contract customers — David and I are still on call if you need us).

Read More

ChangeLog: December 5, 2019 — Student Projects, Repository Config

Welcome back, everyone! We’re here with another ChangeLog, this time focusing on a couple things that just wrapped up: This semester’s student projects, and a series of behind-the-scenes repository configuration improvements.

End of a Semester

Last month, we talked a bit about the CANOSP student program run out of the University of Alberta, and showed off some of the work our CANOSP students have done on Review Board.

They’ve all been hard at work improving our Review UI support (custom review UIs for different types of file attachments), building up both the underlying capabilities of a Review UI and creating prototypes of new UIs for new types of files.

They’ve just wrapped up their semester and completed their final demo videos. We’d like to show off their hard work.

Nicole Hagerman

Nicole’s focus has been the underlying Review UI support, allowing Review UIs to be more dynamic and to not be limited to a single URL. This work has been a backbone of other student projects this semester, so we’re covering it first.

On top of this, she’s built a new Review UI for more easily viewing JSON files, both in their source form and in a structured tree-based form.

Adil Malik

Adil built a series of new Review UIs designed for reviewing:

  • XML files, with options similar to the JSON file Review UI built by Nicole
  • Jupyter Notebooks, a popular tool in the Python world
  • Audio files, complete with waveforms and histograms, offering both diffing and commenting

These have come along really nicely, and show the power of our Review UI support (and the work done by Nicole Hagerman).

Ceegan Hale

Ceegan split his time between some improvements to our diff viewer and to our Review UIs as well:

  • Improved the diff viewer’s display of minified files (e.g., .min.js files)
  • Iterated on our in-progress support for showing Review UIs in the diff viewer
  • Built a prototype Review UI for viewing archive file attachments (e.g., .zip, .rar, etc.).

Repository Configuration Improvements

A good chunk of my own time these past few weeks has been to rework the code behind the repository configuration page. Along with an assortment of bug fixes, we’re working to make it easier to configure plain (non-GitHub/Bitbucket/etc.) repositories.

Historically, plain repositories all shared the same set of configuration fields. You had your “Path” field, “Mirror Path,” “Username,” “Password”. A few had special fields like Perforce’s “Use ticket-based authentication,” but they were baked into the repository form and dealt with specially. Third-party repository support couldn’t add their own fields, and administrators had to translate concepts like a Git Clone URL into our concept of a “Path.”

As of the upcoming Review Board 3.0.16, each type of repository will be able to provide its own configuration form. For instance, down the road, Git repositories will have a dropdown for selecting CGit, GitWeb, etc. as the repository content access method, instead of forcing people to type in a cryptic URL.

Here’s a mockup:

Bottom line: It’s going to be easier to configure repositories in upcoming releases.

There’s a lot of under-the-hood work that’s been done to enable this, and that work is also going to lead to some future improvements we’re looking forward to building in the Review Board 5.0 timeframe. Can’t wait to write about it.

Wrapping Up

That’s another week done. We’ll be back next week!

So what do you think so far? Are the ChangeLogs interesting? Boring? Is there something else you want to hear about? Please let us know on the community forum so we can improve these going forward.

We’re also on Reddit (/r/reviewboard), Twitter, Facebook, and YouTube if you want other ways to keep up-to-date.

Read More

ChangeLog – November 22, 2019 — Review Board 3.0.16 Status

As the holidays quickly approach, we’re trying to wrap up what we can from some of our bigger projects so we can enjoy some much-needed downtime with our loved ones. Thanksgiving is only a week away, and we’ll be taking a short break from ChangeLog in order to focus on stuffing ourselves full of turkey.

So this week will be a but short, but we wanted to go over the next Review Board release on our roadmap, Review Board 3.0.16.

It’s been 5 months!

Yeah, we haven’t had a 3.0.x release in a while. There’s a few reasons for this, and we’ve gone over some of them before. In summary, we’ve been focusing primarily on getting Review Board 4.0 done and adding some much-needed improvements to RBCommons.

Believe it or not, though, there’s active work going into 3.0.16, right now. As in, I’m taking a break from writing 3.0.16 code to write this ChangeLog.

What’s coming?

So there’s an assortment of bug fixes, for sure. Things like:

  • Subversion diff parsing improvements
  • Better bullet-proofing when dealing with truncated Bitbucket webhook payloads
  • Fixes for edge cases where dashboard counters might not update correctly
  • Some search indexing improvements

Along with that, though, we’re making some more important mini-architectural changes:

  • New API for updating a user’s name, e-mail address, and active flag (indicating whether they’re still able to use the server)
  • New API for filtering review requests based on the Branch field
  • Some future-proofing around registration of SCMTools (which handle talking to repositories like Git and Subversion) and HostingServices (which talk to services like GitHub and Bitbucket)
  • Allowing repository configuration to show custom forms for different types of repositories (in-progress)

These are actually pretty important improvements that we wanted to finish before releasing 3.0.16. The API changes are based on a lot of user feedback, and we’re going to finally get this to you soon (sorry for the wait!)

The repository-related functionality is going to not only allow for a better repository configuration experience, but to open the door soon for official support for ClearCase.

ClearCase

Historically, our ClearCase support was entirely community-driven. We were dependent on volunteers to develop and test any fixes or improvements going into that support. This was tough for us, because we know many of you out there do use ClearCase and have had trouble upgrading Review Board due to breakages that we just couldn’t do much about.

This is changing. While it’s still early in development, I’m glad to say we’ll be able to announce a new official ClearCase integration before too long. This will be part of Power Pack and will replace the old, sorta-bitrotted and limited community version.

We’ll have more on this later, but for those of you using ClearCase, we want you to know that improvements are coming. Not for 3.0.16 — again, it’s still early — but before too long.

What about Review Board 4.0?

The last major thing left before beta 1 is a completion of the new administration UI, a topic we’ve also discussed before. This is a big project, required for moving to Django 1.11 and Python 3, but it’s mostly done now. We’re aiming for a 4.0 beta 1 in January, and fast-tracking beta 2 after.

So stay tuned for that after the holidays!

Wrapping Up

That’s it for this week! Again, we’re off next week, probably collectively in food comas. We’ll resume the following week.

If you have any questions, or anything you’re curious about and want us to cover, please reach out on our community forum.

We’re also on Reddit (/r/reviewboard), Twitter, Facebook, and YouTube if you want other ways to keep up with what we’re up to.

Read More

ChangeLog: November 14, 2019 — Student Demos

This week’s ChangeLog is a bit different in focus. Instead of the work we’re doing, we want to talk about some of the work our students are doing.

Want to skip the text but see some cool demos? Scroll down!

Student Programs

For over a decade now, we’ve worked with hundreds of CS students eager to get their feet wet in the industry, mentoring them and giving them cool projects in Review Board to explore. Some have turned into features you use today (such as issue tracking and extensions).

There have been multiple programs over the years that we’ve worked with:

  • Google Summer of Code
  • UCOSP (a Canadian university program we were a part of for many years, now defunct)
  • Open Academy (an experimental university program ran out of Stanford, now defunct)
  • CANOSP (the phoenix rising out of the ashes of UCOSP, currently focused on the University of Alberta)

We’re currently working with CANOSP, piloting the program and mentoring a small group of students as they build some features and prototypes for Review Board.

This Semester

We have three students this semester: Adil Malik, Ceegan Hale, and Nicole Hagerman. They’re all working on various projects that improve the review experience:

  • Review UIs for XML, JSON, and Jupyter Notebooks
  • Hiding the content for minified files in the diff viewer, by default
  • Supporting binary files in the diff viewer

These projects are considered prototypes at this stage. We’re hoping they’ll make it into a release sooner rather than later, but a big part of this work is seeing how feasible these ideas are and what sort of work still needs to be solved before rolling it into production.

XML, JSON, and Jupyter Notebook Review UIs

Adil and Nicole are both primarily focused on the Review UIs, working together on some aspects to beef up the Review UI capabilities under the hood.

Adil built a Review UI for XML files (letting you diff the tree structure, and providing options for changing how that tree is rendered). He’s also been working on a Jupyter Notebook Review UI, playing with ideas for how these would be rendered, diffed, commented on, etc.

Hey, if you’re interested in Jupyter Notebooks, he’s looking for feedback.

Nicole built a Review UI for JSON files, the counterpart to the XML Review UI. To allow for custom rendering options required by both, Nicole’s been building out our baseline Review UI support, allowing them to define utility URLs that can, for instance, re-render parts of content based on the toggle of a checkbox. Adil’s been working with her on the client side of this.

Binary Files and Minified Files in the Diff Viewer

Ceegan’s been building out a feature that we’ve been wanting to bring for a long time. Years back, when we first wrote Review UIs, we intended to use them in the diff viewer so that images, PDFs, etc. that are part of commits could be reviewed without having to upload a separate file attachment.

The base of this work existed in Review Board but was never completed. There were still some missing pieces, problems to solve. That’s what Ceegan’s focusing on: Prototyping the rest of this, getting binary files in the diff viewer working so we can see what’s left.

To get his feet wet, Ceegan built another really useful feature that will make web app developers out there happy. Minified files (*.min.js, *.min.css, etc.) no longer show up as a giant wall of text. Instead, we note to the reviewer that it’s a minifed file and they might not care about it but they can click to see the content. Just like with deleted files.

Demos!

Throughout the semester, we have our students put together demo videos showing off and talking about their work up to that point. We’ve recently completed the Demo 2 milestone, so we have a nice batch of videos to show off.

All videos are uploaded to our YouTube channel. Subscribe to keep up with content as we upload it.

And here are the videos this semester:

Adil Malik

Ceegan Hale

Nicole Hagerman

Wrapping Up

Our students work really hard for their projects, so if you have anything encouraging to say, you’d make their day by saying it on their videos 🙂

We’re hoping to get these in shape to land as part of Review Board 4.0 and 5.0, but it depends on the work that remains once the semester is over.

That’s it for this week, though. If you have any questions, or anything you’re curious about and want us to cover, please reach out on our community forum.

We’re also on reddit /r/reviewboard, Twitter, and Facebook, and of course YouTube if you want other ways to keep up with the latest.

Read More

ChangeLog: November 7, 2019 — Multi-Repo Diffs, Admin UI, and RBCommons

This week in ChangeLog, let’s talk about three projects we’ve been working on this week:

  • Multi-repository diffs
  • The latest administration UI improvements
  • Billing improvements and user roles in RBCommons

Multi-Repository Diffs

It’s pretty common to work with a lot of repositories at the same time, linking them together using Git submodules, SVN externals, or some other method. Sometimes you need to make changes across many repositories at once, and it’d be helpful to post those changes as one review request.

The problem is that diffs made across repositories aren’t always so useful. Not only do they generally lack repository information for each file, but Review Board itself assumes a single repository per review request. This matters because in order to show the diff viewer, Review Board needs to load each source file from the repository before it can apply the diff.

We’re trying to work toward multi-repository diff support as part of Review Board 4.0. Our existing DVCS work got us part-way there. The remaining pieces are/were:

  1. Updating our diff parsers to allow additional information (such as repository details) to be extracted and stored
  2. Updated our FileDiff model (representing a parsed file from a diff) to link to a Repository, and updating code to use that instead of ReviewRequest.repository
  3. Getting this information into diffs

Step 1 is done as of this week, and step 2 is mostly done. Step 3 is where things get interesting.

We talked last week about DiffX, our initiative to make diffs better. We’re hoping to start using DiffX from within RBTools and to inject repository information into the diff.

This is a longer-term goal, though. We’re exploring some options shorter-term, and talking to one vendor about natively providing this information in diffs they generate.

So will this be usable in Review Board 4.0? Kind of. We should have the core functionality all done, but not much may take advantage of this at first. Long-term, we’ll introduce multi-repository diff support for more types of repositories, and it’ll be amazing.

Administration UI Updates

Another week, another batch of screenshots to share of the new administration UI. We’ve completed the main database page and the Change List page, which lists all changes made to the database for a specific type of model (table).

These build upon new CSS components coming to 4.0, which offer slide-out action drawers and filters for datagrids, inline warning/error/info alerts, and more goodies that extension authors can use for their own projects.

Next up are the Settings and Change Form pages, which allow for making changes to models in the database.

Once these are done, we get to upgrade Django!

RBCommons Roles and Billing Updates

RBCommons, our Review Board SaaS, has received a lot of our attention lately. Particularly in the areas of billing, team administration, and sign-up.

Much of this work is based on customer feedback. As RBCommons has grown, our user base has moved from primarily Silicon Valley startups to an international community of companies, organizations, and educational institutions of all sizes. And some of those need a bit more from us when it comes to how they manage their teams and handle billing-related matters.

As part of our plans, we’re working on:

  • Splitting the existing Team Administrator role into three new roles:
    1. Team Owner (capable of making all changes to an account, including cancelling and changing plans)
    2. Team Administrator (capable of managing users, repositories, review groups, etc., but cannot manage anything billing-related)
    3. Billing Contact (can view invoices, change payment information, and receive billing-related e-mails, but nothing else)
  • Improving our invoices, showing more useful information to help users better meet any company-internal, regulatory, or tax requirements they might have
  • Adapting our billing process to work better with credit card billing changes that are or will be going into effect in some countries, to ensure these customers’ payments go through without any headaches
  • Improving the sign-up flow, to help get teams up-and-running a bit faster

We’ll talk about all this in more detail later when we launch our next big update to RBCommons.

Next Week

We’ll have more Administration UI work to show off, and with it a new collection of CSS Components to talk about. We’ll also go over what our CANOSP students from the University of Alberta have been up to.

If you have any questions, or anything you’re curious about and want us to cover, please reach out on our community forum.

We’re also on reddit /r/reviewboard, Twitter, and Facebook, if you want other ways to keep up with the latest.

Read More

ChangeLog: October 31, 2019 — DiffX, CSS Components, Admin UI

This week, we’re going to dive into three major topics:

  • DiffX, our proposed standard for a better Unified Diff format
  • Our new in-house CSS Components standard
  • The work being done on the Review Board Administration UI

DiffX

Unified Diffs, the most common type of diff files, are pretty lacking in standards. Nearly every type of source code management solution outputs its own version of a unified diff, shoving metadata into it however the developers saw fit. This frankly makes it a pain to work with diffs in Review Board, and I’m sure anyone else building similar tools can say the same thing.

A few years ago, we jotted down some ideas of what we’d like to see in diffs. We wanted:

  1. A superset of Unified Diffs that’s fully backwards-compatible (no way would a new diff format gain traction if existing tools couldn’t use it)
  2. A format that could be reliably serialized/deserialized (complete with metadata and file contents) as easily as something like JSON without writing a parser for a dozen different variations of the format
  3. Information on the encodings of a diff file and of the contents of a file, so we no longer have to guess
  4. Changes across multiple commits in a single file
  5. Some standard way of representing changes to binary files

We feel this is important. You wouldn’t believe how wildly different two diff files can be, and how broken most are (just having spaces in a filename means you’re going to run into all sorts of crazy problems).

So we put these ideas together and called it DiffX. Our plan was to build a specification, a standard Python library (and later libraries for other languages), and RBTools/Review Board support.

This week we put up the initial draft of the DiffX specification and documentation. Take a look. See what horrors tools like ours go through, and what we’d like to do about it.

Do you work on another code review tool or source code management system? Let’s talk!

Note that this is still a work-in-progress, and we have a lot we need to better formalize the parsing and the metadata format (and make a public, useful reference implementation).

CSS Components

During Review Board 4.0 development, we kept running into clashes between CSS rules, and had to deeply nest CSS classes and elements in order to ensure our styles would apply. Yuck. This was not sustainable.

We sought to fix this for our code, and looked into what others have done. We considered BEM, BEMIT, and a few other approaches. None felt quite right for our needs (particularly since we want to avoid clashes between third-party extensions).

So we took what we liked in each and drafted our CSS Component Style Guide.

This gives us a better way of thinking about what goes into our stylesheets and HTML pages. We think in terms of formal, reusable components, instead of just haphazardly tacking on styles for whatever HTML we’re writing.

We group together LessCSS variables (used for colors, sizes, etc.) and macros (for customizing a component’s presentation in a safe way) under organized namespaces. Stylesheets (ours and extensions’) that are looking to fit in with our styles or specialize a component can rely on these to better fit in.

We’re also taking care to add formal documentation for everything we add, which forces us to think about our styles, reason about them, and support them.

Want to see this in action? Here’s a few examples:

We’re still working on moving over to this, writing all new CSS using this standard and migrating old styles as needed. This is a long-term project, but we thought it might be interesting to some of you.

If you’re developing extensions for Review Board, we encourage adopting these new styling conventions. It’ll make your life so much easier.

The New Administration UI

Last week, we talked about the work on Python 3 and Django 1.11, and some of the huge tasks we’ve had to work through to support this.

One of the large — and final — hurdles is our administration UI. Django helpfully provides a built-in administration UI that projects get out of the box, and it’s very useful. And we extend and customize it heavily.

The problem is that Django’s administration UI has changed a lot since 1.6, and frankly it wrecked ours. So we’re rebuilding it, better than it was before.

Here’s a breakdown on the new UI:

  • It’s mobile-friendly
  • Organization is now much more clear — all navigation is in the sidebar, not split between sidebar and the top header bar
  • The administration dashboard widgets are so much better — larger, less cluttered, faster, and trimmed down to only the most useful widgets
  • It’s completely our own, no longer dependent on Django’s rendering, so future upgrades will be smoother

It’s also built with our new CSS component standard, and introduces new components that we can use throughout the product. Extensions can take advantage of these, too, helping them look like a native part of Review Board without worrying too much about our styling.

Here’s a couple in-progress screenshots.

Next Week

We’ll go over some of the new diff functionality being worked on in Review Board 4.0, and what we’ve been up to in RBCommons land.

Again, if you have any questions, or anything you’re curious about and want us to cover, please reach out on our community forum.

Read More

ChangeLog: October 24, 2019 – The Python 3 and Django 1.11 Migration

Hi, Christian Hammond here. Welcome back to ChangeLog, where we cover the latest going on in Review Board, RBCommons, and Beanbag in general. It’s been a while since we’ve posted one of these, but we’re aiming to bring this back as a weekly series.

There’s a lot to talk about.

The Short Version

  • We delayed Review Board 4.0 to bring Django 1.11 and Python 3 support. This will not be released by Python 2’s End Of Life date.
  • Python 3 and Django 1.11 support has been in the works for a long time, but is far more complex than it may seem.

The Longer Version: Where are we today

We’ve had four main focuses this year:

  • Getting Review Board 4.0 beta 1 ready to launch. This is way behind our original schedule, but intentionally so — we’ll go over the reasons why in a minute.
  • Bringing integrations options to RBCommons and improving team management, billing, and signup.
  • Building out new functionality for Power Pack (PDF diffing, and in-progress authentication improvements).
  • Growing our business (the prior two tasks were part of that), supporting customers, and assisting some with large projects of their own.

This along with a multitude of smaller tasks taking our time throughout the year (and, on a personal note, dealing with the aftermath of the Paradise, CA “Camp Fire.”)

Most of you are here to find out about Review Board, so let’s dive into that.

The “World Update”: Review Board 4.0, Python 3, and Django 1.11

Our original goal for Review Board 4.0 was to bring support for multi-commit review requests and to ship that. This was a large project, as Review Board was originally written before DVCS was common (Subversion and Perforce were the primarily open source and enterprise solutions).

The problem though, is Python 2 End Of Life is coming up fast, and if we kept on schedule with Review Board 4.0, it would be a long time until we’d support Python 3. So we decided to delay 4.0 in order to get it ready for Python 3.

Note that we also still need to support Python 2.7, since many companies are still heavily tied to 2.7 (older distros, custom extensions and scripts).

So what’s hard about supporting Python 3? Welllll….

The Python Compatibility Problem

Python 3 is leaps and bounds better than Python 2 in most ways, but porting a large and complex product from Python 2 to 3 is even harder than you think.

Most Python users know of the major differences:

  • Modules have moved and functions have been renamed
  • The default string type has changed from byte strings to Unicode
  • Some operators have changed
  • There’s new syntax additions
  • etc.

We thought we prepared years ago to make this move easy. We used Unicode strings in every file. We used the six module to help with using the right modules, types, and functions.

In the end, it was harder than we expected. Our biggest challenge was definitely the Byte Strings vs. Unicode Strings differences. We thought we were in good shape for this, but we weren’t close.

Review Board does a lot of text processing. We’re parsing uploaded diffs, pulling source files from repositories, matching those up and applying the patch to the source, and generating side-by-side diffs. Much of this logic is over 10 years old, even so, we put a substantial amount of work in preparing for Python 3 years ago, so we were shocked by how much we got this wrong.

The problem comes down to the differences in how Python 2 and 3 would handle shoving two different string types together, which is very easy to do accidentally. Python 2 would roll with it, as in many cases the string types were compatible (and would automatically encode/decode so long as the content was basically ASCII). In Python 3, they outright break — which is a better approach, but hard to transition to — and it led us down a rabbit hole of problems.

Missing b'...' prefixes, changes in string return types from Python functions, functions that are no longer compatible with both string types, and very difficult decisions to make regarding compatibility with third-party SCMs.

All the little regressions and inconsistencies added up. Here’s a few more examples:

  • Using Python’s pickle library was a mess. Defaults have changed and pickled data became incompatible (due to module reference and string type changes). We had to build in compatibility layers here and test them thoroughly.
  • Anything that even subtly/unintentionally relied on dictionary sort orders would break.
  • Getting data in/out of processes, streams, and many other objects and functions is now way more sensitive to encoding issues, and required a lot of careful rewrites and testing.
  • Many functions (in Python and third-party modules) that used to return lists now return generators, due to their reliance on other functions that changed return types, and this can cause all sorts of subtle behavioral changes.

We’ve spent a lot of time tracking down issues that might immediately crash or might affect data several stages down, and sometimes be traced to something outside our codebase.

Fixing some Python issues means upgrading third-party modules, which can introduce their own new set of changes, regressions, and new rounds of work.

And the biggest source of that was Django.

The Django Problem

Django 1.6 (the version we’ve been using) breaks on Python 3.6+. This meant we had to upgrade to a newer version, something we’ve been putting off.

Django 1.7 introduced built-in support for database migrations. Quite nice for many projects, but the design was sub-optimal for applications like ours (upgrades could take hours or days longer than our approach) and was fundamentally incompatible with our own Django Evolution.

Many other core components of Django (the foundation for the administration UI, template rendering, URL management, forms, and all sorts of other common and obscure parts) have also largely changed over time in some pretty important ways.

We needed a solution for all of this, and we knew that would take time. To compensate we had to:

  1. Rewrite Django Evolution completely to coordinate evolutions and migrations and support modern Django: ~2 years of work, off and on, with hard problems to ponder
  2. Rewrite our administration UI completely to disconnect from Django’s (ensuring we don’t break again when we move to Django 2.x): estimated ~3 months of steady work, still in progress
  3. Build compatibility layers to help keep older code working and help with new code: ~2 months
  4. Just porting in general, updating dependencies, etc: ~3 months

We’re not done, but we’re close.

The Extensions Problem

Review Board is built to allow third-parties to extend its functionality, modify behavior, and link up with other in-house services. We try to be careful not to break extension functionality, and over time we’ve gotten more strict about providing compatibility and an upgrade path for functionality we want to deprecate or change.

The Python 3 and Django 1.11 upgrade is going to affect just about everybody who’s writing an extension. Some of the work we’ve done on adding compatibility layers will help with this process, but it’s going to mean a longer upgrade cycle for some companies.

We knew from the beginning that we’re going to have our hands full helping these companies out (as part of our support contracts), and that it’d be in our best interest to delay the release and break everything all at once instead of spreading out the breakages across multiple releases, and to also give ourselves time to figure out how best to minimize those breakages.

Most projects moving to Python 3 or to newer versions of Django don’t have to worry about this domino effect in the same way.

Timetable?

I hesitate to say when beta 1 will be done (been wrong before), but I can say that the last major piece of development is wrapping up. We’ll be kicking the tires on it, and want to get a beta out as soon as it’s stable enough.

We may not enable Python 3 builds for beta 1, focusing instead on Django 1.11 testing (Python 3 support is still going to be in development during this time), however we’re working with select customers on real-world testing against Python 3.

Community Questions

Every week, we’d like to address some questions, concerns, suggestions, etc. from the community. If you have any questions for next week, please reach out to us.

Q. The community forum seems quiet at times. Why is that?

A. Support requests from companies with support contracts are conducted over a separate support tracker. We always prioritize these support requests over any other work, and increasingly more companies are moving to this method of support for Review Board.

Q. Will Review Board 4.0 ship with Python 3 before the Python 2 End Of Life date?

A. No, it’s going to miss that date. Python 2.7 is still going to be required for now.

Q. There’s years of open review requests on reviews.reviewboard.org. Why is that?

A. We work with university students every semester to help prepare them for their jobs in the tech industry. They spend their semester building features for Review Board, most of which are prototypes. These make up the majority of the review requests on there. Others are contributions that might be outdated, might have been missed, might be incomplete, or might just not have been reviewed yet.

Next Week

We’ll be going over our new CSS component standard for the project and dive deeper into the new administration UI.

Again, if you have any questions, or anything you’re curious about and want us to cover, please reach out on our community forum.

Read More