Djblets now has a Travis-CI

We used to have a whole, internal automated build system that tested Review Board, Djblets, Django Evolution, etc. against a variety of Python and Django versions. This was responsible for automated tests, nightly builds, website doc generation, and so on.

We’re getting this going again, but this time we’re using Travis-CI. To start off, we’re testing this with Djblets. The testing is minimal, but at least it’s there, and we’ll expand it as we go. Here it is.

Read More

Extensions can now reference static assets from Djblets/Review Board

I’ve made some fixes to the extension packaging today that allow stylesheets in an extension to reference assets (images, fonts, other stylesheets) found in Review Board or Djblets. This wasn’t working before, due to some assumptions Django makes when it comes to post-processing and packaging of static files.

Django allows you to define a list of static namespaces with a given name, and any static path starting with that name is looked up in that resource. So, djblets/images/<blah> or rb/fonts/<blah>.

Works great in templates and everywhere else, except for static file packaging. There, the assumption appears to be that any referenced resources live in the same app. The paths built are always relative to the directory or the top of the static namespace, making it impossible to do the kind of references an extension author would want.

Well, almost impossible. I pretty much beat that into submission. And here it is: https://reviews.reviewboard.org/r/7437/

Read More

Converting database evolutions to Django migrations

May have made some progress toward this today…

When you upgrade Review Board, sometimes it needs to modify your database. This will happen if we’ve added new fields, or tables, or renamed a field, or deleted one, or, you know, databasey things like that. This sort of modification is a database schema migration, and they must be done carefully. They also must support all the databases we support (MySQL, PostgreSQL, SQLite).

Historically, for us, this has been done by Django Evolution. Back before Django 1.7, there was no support for database schema migrations, but some clever people came to the rescue with Django Evolution. We’ve been using it since Review Board 1.0 alpha, and for the past several years, we’ve been maintaining it.

Here’s what it does, in a nutshell:

  • It lets us define what changes need to be made in a database-agnostic way, through “evolution files.”
  • It figures out what changes have already been applied (through a table that tracks those changes) and what changes still need to be applied.
  • It then converts those evolution files into a set of mutation operations, optimizes them, and then converts them to SQL statements suitable to the database.

Now, Django 1.7 finally has support for migrations, but of course the format is wildly different. For in-house projects, this is no big deal. You update the database one last time with Django Evolution, then use Django’s migrations from there on out. That doesn’t work with Review Board, though, since any user can go from any release to any release.

We cannot ever upgrade our Django dependency without solving this problem. Oh yeah, and Review Board extensions are affected as well.

So how do we handle this? Well, this is the game plan:

  • Write a converter to convert the evolution files to Django migration files.
  • Add a hook just before Django’s migration attempt, that imports the Django Evolution history into Django migration’s format. It would do this for any unimported apps, so we need to track those.
  • Write some compatibility shims that convert evolution commands to matching Django migration commands, and begin a deprecation phase.

Seems sorta straight-forward, right? Well, this stuff is always more complicated than it should be, so we’ll see.

I’m on step 1, and so far, this is looking like it’ll work. Expect some long posts with descriptions of how gracefully I’m bashing my head into the wall shortly.

Read More

Djblets now mostly works with Django 1.8!

Djblets is our Django utility library full of reusable components for Review Board. It has datagrids (which powers the dashboard), the API framework, the extension framework, dynamic site configuration, and lots of other utilities.

We just landed a set of changes to bring compatibility with Django 1.8! Let me tell you, going from 1.6 to 1.8 is a huge process, and retaining compatibility with both is not easy. Still, this is a big step.

That’s going into Djblets 0.9, but experimentally. We may actually lock Djblets 0.9 to Django 1.6.x, in order to prevent scenarios where a user upgrades Djblets and it accidentally pulls in the wrong version of Django. We’ll see, though

Also, this does not mean Review Board is anywhere near supporting Django 1.8. That’s a loooong ways off. Review Board 3.0 at the earliest. There’s a lot of work to do there still, and a lot of work to do in Django Evolution to convert to Django’s new migrations support.

(That part is scary.)

Read More

The road to Review Board 2.5

f you’ve been following our development for a while, we’ve been working on a big 2.5 release. This started off as a nice little feature release that was going to focus on API tokens, Webhooks, and a handful of other features, but has since evolved to include a UI refresh and preliminary mobile support.

It’s taken longer than we’d like, surely. That happens in software, and we’re still a pretty small team with very full plates. The good news is, we’re getting pretty close to release.

So here’s where we are now:

  • We’re feature-frozen, but still polishing things.
  • We’re fixing bugs here and there (though things are looking quite stable so far).
  • Getting ready to release beta 2 (soon!)
  • Followed by a RC release, and then the final 2.5!

Awesome. So, we’re getting there. Follow us on our ChangeLog, and you’ll get to watch as we get closer and closer to the final 2.5 release.

Read More

RBTools 0.7.3 is out, with Unicode fixes and more

Have you been plagued by Unicode errors when posting changes? Subversion 1.7.x compatibility problems? SSL errors with self-signed certs on Python 2.7.9+? Well then, today’s release of RBTools 0.7.3 is for you! … and, well, everyone, really. There’s a lot in here you’re going to want.

First off, the Unicode fixes. We had some Unicode-related breakages in past releases, which have been almost entirely eliminated in this release. If you’ve hit any such error before, give 0.7.3 a try.

There are lots of compatibility and behavioral improvements for Bazaar, ClearCase, CVS, Subversion, Perforce, Plastic, and TFS. Patching, for instance, works a lot more reliably across the board. Subversion 1.7.x and lower should start working again as well.

It’s not all bug fixes, though. We have some new features and command line options:

  • rbt post --stamp will auto-stamp commits with the review request URL.
  • rbt patch -R can be used to revert a patch from a review request.
  • rbt land and rbt stamp now work with Perforce.
  • Perforce supports changesets with imported files from remote depots.
  • API caching behavior and cookie storage can now be disabled/changed for any command. Useful for hooks and other scripts.
  • SSL certificate verification can be disabled on Python 2.7.9+ by using --disable-ssl-verification.

Those are the highlights! For the complete list of changes, see the release notes.

But wait, we have a couple more announcements concerning our releases:

Better installation through pip and Wheels

We’re now officially releasing Python Wheel packages for RBTools, supporting the latest versions of pip. To install RBTools, simply type:

$ pip install --allow-all-external RBTools

(In the future, we’re hoping to eliminate the need for --allow-all-external.)

Signed releases

As of this release, we’re now signing all builds with our official PGP key. Using gpg, you can verify a build was produced by us by importing our public key and then verifying it against the .asc signature files from our downloads.

For example, to verify RBTools-0.7.3.exe was produced by us, download it along with RBTools-0.7.3.exe.asc and run:

$ gpg2 --recv-key 4ED1F993
$ gpg2 --verify RBTools-0.7.3.exe.asc

(Note that if it complains about the key not being certified with a trusted signature, it’s just because you haven’t signed our key. If you see that message, and the primary key fingerprint is 09D5 06DA BB62 A09E 891D A9F3 2852 91B3 4ED1 F993, it’s good!)

We’ll have documentation covering this in more detail soon.

Thanks everyone, and as always, feel free to reach out with any questions.

Read More

RBTools 0.7.2 is here!

It’s been just over a month since RBTools 0.7.1 arrived at stores near you (or probably not). That’s a whole month spent not upgrading RBTools, and we can’t have that. Plus, there were some good bug fixes and new commands we thought we’d send your way. So let’s not waste any more time. Here’s what’s in today’s release of RBTools 0.7.2.

We’ve fixed some issues working with Bitbucket/Beanstalk/Unfuddle. We’ve heard reports from users that posting changes against their Git repositories hosted on these services that just resulted in failure, so we’ve gone ahead and fixed that all up.

There are also two new RBTools commands for you to play with:

  • rbt login will log into RBCommons and store the cookie. rbt post and other commands will still do that, but it’s sometimes handy to have new users log in once in order to ensure they’re all set up properly.
  • rbt logout will log you out of your API session on RBCommons, and delete the cookie.

For the full list of changes, see our release notes.

Read More

Better branch navigation with git-j

One of Git’s core strengths are its simple, light-weight branches. Spend any time working with Git and you’ll soon develop the habit of creating feature branches for any change. As your project grows, you may start to introduce release branches, hotfix branches, and long-running feature branches.

Navigating between branches becomes a common task. A simple git checkout <branchname> is all it takes. Not so bad, but if you’re frequently switching between branches, all the typing can add up.

That’s where git-j comes in.

git j is not only faster to type than git checkout, but it also offers some quick shortcuts for fast branch navigation, making it just as easy as directory navigation.

Here, I’ll show you.

Go up a branch with git j ..

Much like directory navigation, git j .. jumps to the nearest parent branch.

Let’s say you have this branch scheme:

o [topicB] [HEAD]
|
o [topicA]
|
o [master]
|

Say you want to go down a level to topicA, do some work, and then go down again to master. Using git j ..:

$ git j ..
# Do some work
$ git j ..

Now you’re on master. Check out the savings compared to git checkout:

$ git checkout topicA
# Do some work
$ git checkout master

Hey look, you’ve saved 22 keystrokes!

Jump to your previous branch with git j -

Ever find yourself switching back-and-forth between two branches? We have a handy shortcut for that.

Let’s go back to our branch scheme from before. You’re on topicB, and you need to go down a branch to topicA, do some work, and jump back to topicB.

Simple.

$ git j ..
# Do some work
$ git j -

Much nicer than the alternative:

$ git checkout topicA
# Do some work
$ git checkout topicB

That’s another 23 keystrokes saved.

Use aliases for common branches

You probably have a few branches you’re frequently on, and I bet one of them is master. Let’s say that another is release-1.0.x. Oh, let’s also throw in something like 2.0/big-refactor on top of that.

How often are you typing those branch names? Every typo any of them? I sure have.

We can shorten all those names with git j alias. Let’s give them nice, short, easy-to-remember names:

$ git j alias m=master
$ git j alias 1.0=release-1.0.x
$ git j alias rf=2.0/big-refactor`

Now instead of typing those branch names, all you have to do is pass the aliases to git j, like so:

$ git j m
$ git j 1.0
$ git j rf

Much nicer than:

$ git checkout master
$ git cehckout release-1.0.x
$ git checkout 2.0/big-refactor

Wouldn’t you say?

I use that m alias all the time, since practically all Git repositories have a master branch. Instead of setting up an alias for all of them, I can make a global alias:

$ git j alias -g m=master

Now git j m will work wherever I go!

To unset an alias, just pass an empty branch name:

$ git j alias rf=

By the way, if you ever need to check which aliases you’ve set up, just simply type git j aliases. For global aliases, git j aliases -g.

Working with branch history

git j keeps track of which branches you’ve checked out most recently, and makes it easy to jump between them.

Simply run git j history to see what your branch history looks like.

Want to quickly jump to a branch in your history? git j <number> is all you need. For instance, git j 2 will jump 2 branches back in your history.

Putting it all together

Okay, here’s a big, real-world-ish example. Our Review Board repository has master and release-2.0.x branches, and I’ve introduced a my-feature branch. In this example, I’m going to:

  1. Work on release-2.0.x.
  2. Rebase my-feature on top of it.
  3. Merge it into release-2.0.x.
  4. Merge the latest release-2.0.x changes into master.
  5. Go back to working on release-2.0.x.

First, here’s how you’d traditionally do this with Git:

# 1. Work on release-2.0.x.
$ git checkout release-2.0.x

# 2. Rebase my-feature on top of it.
$ git checkout my-feature
$ git rebase release-2.0.x

# 3. Merge it into release-2.0.x.
$ git checkout release-2.0.x
$ git merge my-feature

# 4. Merge the latest release-2.0.x changes into master.
$ git checkout master
$ git merge release-2.0.x

# 5. Go back to working on release-2.0.x.
$ git checkout release-2.0.x

Now let’s try it with git j, and with a couple of aliases (m for master, 2.0 for release-2.0.x):

# 1. Work on release-2.0.x.
$ git j 2.0

# 2. Rebase my-feature on top of it.
$ git j my-feature
$ git rebase release-2.0.x

# 3. Merge it into release-2.0.x.
$ git j ..
$ git merge my-feature

# 4. Merge the latest release-2.0.x changes into master.
$ git j m
$ git merge release-2.0.x

# 5. Go back to working on release-2.0.x.
$ git j -

How’s that for less typing? As you start to work with git-j, it’ll all start feeling more natural, just like walking a filesystem.

Get started with git-j today!

Simply clone our dev-goodies repository and stick dev-goodies/bin/git-j somewhere in your path. Or, stick all of dev-goodies/bin/ in your path to get easy access to all our scripts.

We’ll talk more next week about another extremely useful git script we provide called git-rebase-chain. Stay tuned!

Read More

RBTools 0.7.1 is released

Today’s release of RBTools 0.7.1 fixes up several issues found in this month’s big 0.7 release, and helps to increase performance when posting changes for review. There are also compatibility fixes for Windows, and improvements to the new Windows installer.

Some of the highlights include:

  • Unicode compatibility fixes when running on Windows.
  • CVS/Subversion diffs are once again generated from the current directory and not the root of the checkout.
  • Several fixes for the Windows Installer.
  • Support for invoking RBTools in a Bash shell on Windows.
  • Fewer API requests are needed to post changes for review.

For the full list of changes, see the release notes.

Read More