Beanbag’s Best Bad Bugs of February 2016

We’re kicking off a new series here at Beanbag Inc., makers of the popular enterprise collaborative software review tool Review Board.

Beanbag’s Best Bad Bugs highlights the consequences of missing bugs and vulnerabilities before code goes into production.

Before diving into the inaugural list, we want to clarify a few things:

  • Nobody is perfect! We make code review software and even we don’t catch every bug before we go live — it happens! The truth is that with today’s complex deployment models, massive growth in apps and code and ever-growing number of dependencies, it’s nigh impossible to foresee everything prior to shipping. In our view, this makes rigorous, yet efficient, code review more important than ever.
  • The point here is to raise awareness to the need for peer code review by pointing out — with a little levity — where, as an industry, we missed some opportunities.
  • We’ve assigned each Bad Bug a severity rating from 1 (annoying but not damaging) to 5 (very dangerous). This is just our opinion.

Without further ado, let’s get to February’s list!

10. No Visual Studio for YOU!

When you depend on an app and it goes down, life really sucks. That’s what happened to users of Visual Studio Online in February when the site went dark for 5 hours.

The culprit: According to Microsoft:

A SQL stored procedure that was being called was allocating too much memory in one of the critical backend SQL databases. After an extended period of time, this caused the SQL databases to fall into an unresponsive state and resulted in customers being unable to access their VSTS accounts.

Our rating: Bug

Rationale: So, affected teams basically experienced a productivity drain similar to that of every company in a major college basketball town during March Madness. Bummer? Yes. Catastrophe? Not even close. As the article points out, the bigger issue might be how this outage reflects on Microsoft’s overall cloud image.

9. It’s getting hot in here!

A few unlucky customers of the British Gas Hive home automation device and app found it just a tad toasty, to put it with English understatement. Yeah — their thermostat got pegged at 32°C — for us Yanks, that’s just shy of 90°F.

The culprit: According to British Gas:

We are aware of a temporary glitch affecting a very small number of customers, where a certain sequence of commands in the Hive iOS app can cause the thermostat temperature to rise to 32°C.

Weird, but OK.

Our rating: Bug

Rationale: Buyers of Smart Home gizmos right now epitomize early adopters — so they’re likely to be totally cool (ahem) with a bug or two in exchange for the “first on the block” factor. Not to worry British Gas, but don’t make it a habit unless your true aim is to short the entire textile sector.

8. Nest takes home temps to the other extreme.

So many musical references come to mind with these polar opposite smart home foibles — we’ve got Alphabet’s Nest whose battery could lose life, leaving affected customers shivering.

The culprit: The NY Times reports:

“Matt Rogers, the co-founder and vice president for engineering at Nest, blamed a software update from December. “We had a bug that was introduced in the software update that didn’t show up for about two weeks,” Mr. Rogers said apologetically. In January, devices went offline, and “that’s when things started to heat up.”

Our rating: BugBug

Rationale: I know what you’re thinking — whoa — hold on, 1 bug for Hive and 2 for Nest for essentially identical issues? Double standard!

Let me explain.

Alphabet / Nest should be held to a higher standard, in part because they are Google and also because Nest goes beyond temperature to include control of things like smoke alarms and home security, where the stakes are much higher.

To their credit, all reports indicate they busted their tails to make things right for affected customers, but they aren’t likely to achieve their expected share of this growing market if these sorts of mishaps continue.

7. One ringy dingy.

Keeping with our IoT and smart homes theme, Ring (clever name btw) shocked users with a major vulnerability that gave would-be hackers the customers’ Wi-Fi password at the push of a button.

Now, I do think the tech press loves to make hay out of all such stories. In order to actually get the Wi-Fi password, you have to remove the doorbell from the house and press the orange button on the back — not something most hackers would have known to do, even if they could identify a Ring doorbell from the street.

The culprit: Just bad design. The company fixed the issue and all’s right with the world.

Our rating: BugBug

Rationale: Identify theft is really scary and once a hacker has access to your Wi-Fi network, they can potentially access all the info they need to destroy your credit, or worse. So, while the chances are small that a hacker would know that you have a Ring doorbell and that by detaching it they could get your Wi-Fi password, the potential impact is significant.

6. I’m going to need you to go ahead and stay in the office, mmm’k? That would be great.

Commuters tend to not be the happiest of campers to begin with. Daily they face clogged roads, other drivers looking at their phones instead of the light that just turned green, trains that sometimes run on time, but often don’t and countless other indignities.

Enter Bug #6 in our list. According to the English tech site The Inquirer, several travel apps that London commuters use to track the status of Tube lines indicated that every single line was closed at the height of rush hour.

Take a look.

tube_down

The culprit: All Transport for London had to say was that “a bug” caused the faulty information. The way their system works, all the different tube status apps pull info from a central feed and it was here that the bug manifested itself.

Our rating: BugBug

Rationale: Toyed with giving it a 1 but, just because commuting sucks so much to begin with, felt this particular snafu deserved a 2.

Editor’s Note: This blog was drafted a few weeks ago — our hearts go out to Belgium. Clearly with security on everyone’s minds, one can see how a bug such as this could produce significant unwarranted anxiety.

5. Airlines. ‘Nuf said.

Delta was the latest airline to experience an operations disruption due to software failure. In Delta’s case, a ground operations app that stopped working delayed boarding for about 25 flights.

The culprit: Airlines don’t tell you anything about your flight status or what caused their software to go down. They didn’t comment on the cause of this outage, but assured us they’ve got it under control and it won’t happen again. OK.

Our rating: BugBug

Rationale: Sure, it was only a handful of flights and all this outage did was delay boarding, but airlines have racked up so much ill will with customers that they really have to prove that they care, again IMHO. So that’s why 2 bugs.

4. Creative Cloud Party Crasher.

Let’s shift from IoT to cloud-based productivity apps. As a marketer, this one is near and dear to me. Earlier this month, Adobe faced the wrath of not just marketers, but MacBook-using marketers that use their Creative Cloud, which up and decided to delete the contents of the first folder to show up alphabetically in a user’s root directory.

The mess up hit users of the popular backup tool Backblaze particularly hard. In addition to deleting folders, it also froze users’ back-up capability. Jeesh!

The culprit: A bad update — specifically 3.5.0.206 — contained a “rogue script” that carried out the carnage. Rogue script? One wants more details…

Our rating: BugBugBug

Rationale: Deleting marketers’ folders is going to take a while for Adobe to recover from, IMO. Sure, it’s not as severe as identity theft, but if I lost an entire folder of work, and also the ability to back things up, I’d be really mad. And with software makers pestering customers to upgrade the way they do – and for good reason, since many upgrades are designed to protect customers from new vulnerabilities – it’s really incumbent on vendors to make sure their updates are safe.

3. Would you like privacy with your app? Oh, sorry, looks like we’re all out of privacy.

Here’s a mind-blower — Baidu, the giant Chinese app-maker, has been scarfing up users’ personal information left and right for “commercial” use, saying that they “only provide what data is lawfully requested by duly constituted law enforcement agencies.” Riiiight.

It’s highly doubtful that even a faithfully implemented peer code review process would stop this behavior.

The culprit: According to Reuters:

The researchers at Canada-based Citizen Lab said they found the problems in an Android software development kit developed by Baidu. These affected Baidu’s mobile browser and apps developed by Baidu and other firms using the same kit. Baidu’s Windows browser was also affected, they said.

Our rating: BugBugBugBug

Rationale: Not all countries value customer privacy and freedom the same, but the world’s getting smaller by the day, and these sorts of problems aren’t isolated to just a handful of countries anymore.

2. Wait a sec — I didn’t turn on remote control!

If you were using the NissanConnect EV app with your Leaf, yea, you kinda did. Researchers figured out that they could use the app to hack into a Leaf’s APIs and, with an anonymous GET request, access all kinds of information from the car’s systems, like trips, location and more.

When reading about this app (Nissan has since disabled it) and the nonchalant way it handed over the information about the car to anyone, the details of what the hacker can access fade to the background. Sure, in this case, the hacker can’t access the car’s operational systems. But one certainly hopes this serves as a big old wake up call to all automakers rushing to tap the power of the Internet, software and APIs to deliver cool new features.

The culprit: This one really looks and smells like carelessness. One would think that even the most casual internal peer review on this app and API functionality would have surfaced this issue.

Our rating: BugBugBugBug

Rationale: Nissan is kind of taking one for the automotive team here. The prospect of a hacker accessing automobiles is pretty scary and needs to be prevented by higher coding and review standards.

1. Give me your money, and your identity!

This isn’t a new exploit, but the IRS data breach just keeps getting worse. In February, it was reported that the total number of impacted citizens could be 700,000! Talk about adding insult to injury.

The culprit: As was widely reported in May when the vulnerability and data loss first came to light, the issue was with the Get Transcript feature of the IRS web site, which revealed EVERYTHING to the hackers — income, address, SSN, you name it. Hackers were able to get past the Knowledge-Based Authentication (KBA) — you know those security questions like what street did you grow up on — by using information they stole from other sources. Once in, they literally had access to people’s entire tax returns.

In a sadly ironic new twist, the IRS distributed PINs to all the breach victims, but if you forgot your PIN, the IRS left the same KBA system to fence off your data. At least one victim found that her PIN had been compromised, no doubt by hackers with access to the security question answers that allowed her records to be breached in the first place. Just wow!

Our rating: BugBugBugBugBug

Rationale: If it can get worse than this, I really don’t want to imagine what that looks like — 5 out of 5.

The bottom line…

With all of these bugs, crashes and hacks, implementing the right tools and processes now can save you money and/or your reputation later. You won’t always catch everything, and code review does take additional time up-front, but the savings in the long-run is totally worth it.

We hope you enjoyed this inaugural Beanbag’s Best Bad Bugs. If you’ve got ideas for future lists, send them our way to bestbadbugs@beanbaginc.com or drop them in a comment.

Read More

Announcing virtualenv-multiver for Python Development

If you’ve worked with virtualenvs for Python before for development/testing, then you may have hit cases where you really wanted multiple versions of Python installed in your virtualenv. Which, you may actually have working, because virtualenv, in theory, supports this. In fact, you’re supposed to be able to do:

$ virtualenv -p python2.6 my-env
$ virtualenv -p python2.7 my-env

That’d be great, if it always worked. It doesn’t. When your virtualenv gets built, bin/python may end up being a link to bin/python2.7 (or what have you), or it may be the contents instead of a link. Subsequent installs may end up overwriting binaries, producing a python2.6 and python2.7that are both Python 2.7.

Oh and it gets worse. On Mac, with a standard Python install, these binaries actually end up invoking ../.Python, a symlink pointing to the system Python. This link is not versioned. So much for multiple Python versions in one virtualenv on the Mac.

A solution!

We fixed this. Now you can run a single command to get a working environment going, without messing with things or running into problems on the Mac. This is virtualenv-multiver.

Now, setting up an environment is as simple as:

$ pip install virtualenv-multiver
$ virtualenv-multiver my-env 2.6 2.7

Couldn’t be easier. This works both for new environments and existing ones.

This is a beta, so there may be some issues here or there. If this is useful to you, give it a try and let us know!

Read More

RBTools 0.7.5 is here!

RBTools 0.7.5 is now out and ready to install.

This is largely a bug fix release, focusing in part on improved compatibility with Windows, Git, Subversion, Mercurial, Perforce, and Team Foundation Server.

On Windows, RBTools will now first look in %HOME% to find any custom .reviewboardrc files, instead of only looking in the Application Data directory, which will be quite helpful with many system configurations. There are also fixes for using Mercurial on Windows.

Non-Git user? You’ve probably seen that annoying but harmless command not found: git error when posting a change. That’s gone now!

For Perforce users, posting submitted changes or files outside of the client view now work. This had regressed in an earlier release, but you should be in good shape now.

Subversion has seen some more Unicode fixes, plus fixes for rbt post --svn-show-copies-as-adds.

Along with all this, we’ve added a new feature for setting a custom search path for .reviewboardrc. You can set your $RBTOOLS_CONFIG_PATH to a list of paths to search, allowing you to make your version in $HOME take precedence over what’s in your repository, and allowing you to work with centralized collections of aliases in your organization.

See the release notes for the complete list of changes.

One more thing: We’ve simplified installation for those of you using pip to install. Our builds are now directly hosted on PyPI, meaning all you now need to do to upgrade is run pip install -U RBTools.

Read More

Introducing new special user permissions

As a team grows, it often becomes the case that more developers need to assume more specialized roles in the code review process. Not just that of developer and reviewer, but also that of a manager of sorts, helping to keep the review process going and to keep the process tidy.

We’re introducing a few new special user permissions, designed to give users a subset of an administrator’s abilities. These can all be set in the Team Administration page by clicking the pencil icon next to team member.

 

 

The first permission, “Can close or reopen review requests from other users,” enables a user to help keep the list of review requests tidy by toggling whether a review request is currently open. If you’re not auto-closing review requests, if you have review requests open from former team members, or if you’re managing an open source project on RBCommons, this can be quite handy.

The second permission, “Can edit review requests from other users,” allows a user to modify a review request on someone else’s behalf. They can upload diffs, edit fields, and so on. The changes currently appear as if they’re from the owner of the review request.

The final permission, “Can post review requests as other users,” is most useful for scripts. In cooperation with RBTools (using rbt post –submit-as), a script can post a review request on another user’s behalf, perhaps when a change is committed to a special branch, or after a sandbox operation passes.

We’ve been piloting these permissions with some projects for a while now. Please let us know how they work for you, and if you have any questions or problems.

Read More

Work toward a Django 1.8+ port for Review Board

We’ve been dependent on Django 1.6 for our components, for many reasons. There are a lot of things Review Board has to deal with that most Django apps do not, so Django often regresses us, unintentionally. There are also just design changes in Django that don’t suit shipping products, and we’re often having to work around these changes.

However, the biggest bit is our database migration story. Our codebase depends on Django Evolution for migrations, which has to dive pretty far into the internals of Django for some operations. This is a large bit of work to port.

I’m happy to say that, after spending the day on it, I got surprisingly far toward having Django Evolution working on Django 1.8! It’s not perfect, and there’s not yet a good story for playing nice with Django migrations, but it’s a huge start. It opens the doors to getting a full compatibility story going.

The next question is, are we keeping Django Evolution, or moving to migrations fully? Well, that’s a bigger problem, because we have no control over which version of Review Board (and therefore Django) people are upgrading from, and have to be very careful with how we handle any database migrations.

There are also issues with Django’s migrations just being a lot slower than Django Evolution, to the intentional lack of an optimization step when applying the migrations. This means stupidly-long upgrades for large installs, which won’t work for us. So, we’ll probably stay with Django Evolution until we figure out a decent solution there…

Read More

Plans for Review Board’s bug tracker

Many of you may have heard that Google Code is going read-only starting tomorrow, and some have asked us how this will affect the project, since we host our bug tracker there.

Not to worry. Google’s been nice enough to whitelist us for a little while, so even though most of Google Code will be down, we’ll continue to be up. This is not permanent, but for the time-being, you’ll still be able to report bugs at the old address.

Going forward, we’ll be migrating off of Google Code and onto a new tracker. That will happen in the coming weeks, and we’ll talk more about it when it happens.

So why the delay? Why did Google need to extend the shutdown date for us? We actually have something new on the way that we’re pretty excited about. We call it Splat, and while still very young, it’s shaping up to a pretty cool bug/issue tracker. We weren’t quite prepared to switch over to it by the shutdown date, but we have enough of it ready to launch pretty soon.

There’s a lot more that I’d like to say about Splat, but there will be time for that. We’ll make a more formal announcement soon.

Read More

Lots of UI cleanups in preparation for Review Board 2.5 RC 1

2.5 beta 2 looks to be working pretty well, and we’re working hard to get ready for RC 1. As part of this, we’ve fixed up a number of little UI issues here and there. For instance, login on mobile now works:

 

Mobile Login (Review Board 2.5)

 

As does registration and password resets.

Gravatars are now showing up more reliably in the dashboard. Depending on the settings on the server, these may have been hidden unintentionally. Basically, defaults weren’t being taken into consideration in some calls.

The user page now works properly on mobile, with filters moving to a little menu:

 

Mobile Dashboard (Review Board 2.5)

Mobile Dashboard with Filters (Review Board 2.5)

 

Also, some fixes for visual issues with text and Markdown file attachment review pages.

Read More

On-the-fly syntax highlighting when using Markdown

Review Board 2.0 introduced Markdown support for text fields, and we’ve been iterating on this since. One nice advantage to using Markdown is that it’s really easy to syntax-highlight a code fragment, like:

```python
def foo():
print "oh hi there"
```

When saving the comment, this would appear rendered with some syntax highlighting, same as the diff viewer.

In 2.5, we’re adding on-the-fly syntax highlighting for most popular languages: CoffeeScript, CSS, Go, HTML, JavaScript, Perl, PHP, Python, ReStructuredText, Ruby, Shell Scripts, SQL, XML, and YAML.

That means when you type code in a code block, like above, it will show the syntax highlighting immediately, without having to save.

Now, it’s not perfect. We use a different highlighting engine for rendered content vs. on-the-fly content, and they don’t 100% agree on how things should be styled, but it’s close enough.

We’re gaining this ability through an upgrade of CodeMirror, the widget we use for the text fields. We’re giving 5.5 a try (we previously tried upgrading to 4.2 and had issues, but so far so good with 5.5).

This will all ship with Review Board 2.5 RC 1.

Read More

A new polished issue summary table for review requests

Since Review Board 1.6, we’ve had a table of all open issues (comments that have a task that needs fixing before a change can go in), sitting right below the fields of a review request. This could be filtered by reviewer and by status type, showed summaries of the comments, the date/time the comment was filed or last updated, and the status type.

It was pretty text-heavy, though, and not easy to read at a glance. As such, most people probably ignored the nice filtering abilities and, really, most of the content.

So, I redesigned it.

Here’s how it looked before:

Old issue summary table

And here’s how it’ll look in Review Board 2.5 beta 2:

New issue summary table (Review Board 2.5)

New issue summary table (Review Board 2.5)

It’s even more mobile-friendly:

Mobile issue summary table (Review Board 2.5)

Users will get to use the new table later this week.

Read More

Using Review Board with Amazon CodeCommit

Today, Amazon released their all-new CodeCommit service as part of the Amazon Web Services family. CodeCommit is a Git repository hosting service built for scalability and reliability, helping to securely store encrypted versions of your code, binaries, and configuration related to your products and cloud infrastructure.

They’ve put together a guide on integrating AWS CodeCommit with Review Board that you can follow if you’re wanting to give this service a try. It’ll walk you through deploying a Review Board server, setting up access to CodeCommit, linking your repository, and posting changes for review.

Currently, setup requires maintaining an in-sync clone of your repository on the Review Board server. We’re aiming to work with the CodeCommit team to help bring direct support for hosted CodeCommit repositories to a future release of Review Board and RBCommons.

For more information on getting set up, check out the CodeCommit page and read our guides on configuring Git repositories and our recommended RBTools workflows for Git.

Read More