May have made some progress toward this today…
When you upgrade Review Board, sometimes it needs to modify your database. This will happen if we’ve added new fields, or tables, or renamed a field, or deleted one, or, you know, databasey things like that. This sort of modification is a database schema migration, and they must be done carefully. They also must support all the databases we support (MySQL, PostgreSQL, SQLite).
Historically, for us, this has been done by Django Evolution. Back before Django 1.7, there was no support for database schema migrations, but some clever people came to the rescue with Django Evolution. We’ve been using it since Review Board 1.0 alpha, and for the past several years, we’ve been maintaining it.
Here’s what it does, in a nutshell:
- It lets us define what changes need to be made in a database-agnostic way, through “evolution files.”
- It figures out what changes have already been applied (through a table that tracks those changes) and what changes still need to be applied.
- It then converts those evolution files into a set of mutation operations, optimizes them, and then converts them to SQL statements suitable to the database.
Now, Django 1.7 finally has support for migrations, but of course the format is wildly different. For in-house projects, this is no big deal. You update the database one last time with Django Evolution, then use Django’s migrations from there on out. That doesn’t work with Review Board, though, since any user can go from any release to any release.
We cannot ever upgrade our Django dependency without solving this problem. Oh yeah, and Review Board extensions are affected as well.
So how do we handle this? Well, this is the game plan:
- Write a converter to convert the evolution files to Django migration files.
- Add a hook just before Django’s migration attempt, that imports the Django Evolution history into Django migration’s format. It would do this for any unimported apps, so we need to track those.
- Write some compatibility shims that convert evolution commands to matching Django migration commands, and begin a deprecation phase.
Seems sorta straight-forward, right? Well, this stuff is always more complicated than it should be, so we’ll see.
I’m on step 1, and so far, this is looking like it’ll work. Expect some long posts with descriptions of how gracefully I’m bashing my head into the wall shortly.
