Announcing kgb 2.0 for Python – Function spies for unit tests

We’ve just released a new major version of kgb, a Python library for creating function spies in unit tests. This is a very handy tool for helping craft unit tests in Python applications.

kgb 2.0 introduces support for Python 3.6, improves argument checking, and removes the need for a special .spy attribute on standard functions.

What are function spies?

Function spies allow you to listen for when functions are called, what parameters they were passed, what value they returned or exception they raised, and allow you to disable the function’s normal behavior and optionally replace it with your own. This is a popular feature of Jasmine, a testing framework for JavaScript.

They’re particularly useful when working with third-party libraries whose behavior you cannot normally change. For example, your project might call a function in a library that in turn calls out to a HTTP server, which might be problematic for your unit test. With kgb, you can simply spy on urllib2.urlopen and return a custom result.

For example:

import logging
from unittest import TestCase
from urllib2 import urlopen

from kgb import SpyAgency


class MyTests(SpyAgency, TestCase):
    def test_http_request(self):
        def _fake_urlopen(opener, *args, **kwargs):
            self.assertEqual(url.get_full_url(),
                             'https://example.com/123/')

            class FakeResult(object):
                def read(self):
                    return 'Your fake payload goes here!'

            return _FakeResult()

        self.spy_on(urlopen, call_fake=_fake_urlopen)

        # Imagine that this function makes an HTTP request to
        # https://example.com/123/ and logs a message.
        some_function_that_does_urlopen()

        self.assertTrue(urlopen.called)
        self.assertTrue(logging.info.called_with(
            'Fetching something from an API'))

What’s new in kgb 2.0?

Better, more consistent spies

We removed the distinction between spying on standard functions and methods on classes. This used to be treated differently. Previously, you could call spy functions like .called_with() and access attributes like .last_call directly on the method, but for functions, you had to use .spy.called_with() and .spy.last_call. We also kept plain functions mostly intact, but replaced methods on a class with a method-like object designed to intercept calls and mimic the method’s signature. That meant things were different depending on what you were spying on.

We now keep the methods where they are, bringing the spy functions and attributes onto the spied functions directly. We also use a special bytecode injection process for all spying operations (it’s very complicated, but awesome).

What does this ultimately mean? Well, it means if you had code from older versions that looked like this:

self.assertTrue(my_func.spy.called)

You can trim off the .spy part:

self.assertTrue(my_func.called)

It also means that we’re spying at a lower level than before for methods on classes, helping to prevent problems with code that’s sensitive to methods being replaced.

And it gives us Python 3.6 support.

Python 3.6 and PyPy support

See, there it is!

kgb 2.0 now fully works with Python 3.6. And as a bonus, PyPy as well.

More flexible and descriptive argument checks

called_with() now lets you check positional arguments by specifying their names as keyword arguments, and doesn’t require that you check for all arguments passed to the function. For example, if you have this method:

def my_func(a, b, c=123):
    pass

you can inspect the calls with:

self.assertTrue(my_func.called_with(b=True))

Hand-holding when things go wrong

If something goes wrong in your test suite and your spy fails to unregister at the end of the test, you could get some pretty confusing assertion errors in older versions of kgb. Same if you accidentally try spying on the same function twice in your tests.

In kgb 2.0, we check for this and present a very helpful error showing you exactly where the spy was originally set so you don’t have to hunt it down yourself.

Learn more about kgb

We’re biased, but this is a really nifty library, and has made our lives so much easier. We have full documentation up on GitHub showing all the ways you can work with spies, along with a FAQ.

Installation is easy. Just run:

$ pip install kgb

kgb supports Python 2.6 through 2.7 and 3.4 through 3.6, along with new, experimental support for PyPy.

If you find kgb useful, please tell others about it, and give it a star on GitHub.

Read More

RBCommons updates have moved to the Beanbag Blog

For years, we’ve been maintaining three separate blogs for our products: the RBCommons Blog,¬†Review Board News, and the Beanbag Blog. It made sense at the time to keep these separate, but these days it’s usually more confusing than it needs to be, with release announcements and helpful guides scattered across the blogs.

We began the process of consolidating these last night, and started with merging the RBCommons Blog into the Beanbag Blog. Unfortunately, due to a glitch with our mailing list provider, an e-mail went out today covering last February’s CloudFlare-related security issue. If you received this, we’re very sorry — that shouldn’t have happened, and you don’t need to worry about some new problem affecting RBCommons.

We’ll be posting more articles here going forward, along with RBCommons updates and RBTools release announcements. We recently started a series of articles on new Review Board features that will soon make its way to RBCommons as part of a major update we’re gearing up for.

We’re also planning to move the Review Board release announcements here, so there’s exactly one place to look for everything we’re working on.

And with that, we’d like to thank you all for being such wonderful customers. Have a Happy New Year, everyone! Here’s to a great 2018 ūüôā

Read More

RBTools 0.7.10 is now out

Today’s release of RBTools 0.7.10 some important compatibility fixes for macOS, Git, Subversion, Team Foundation Server, ClearCase.

macOS and Browser Windows

macOS users who have upgraded to recent releases of Sierra lost the ability to run rbt post --open (to open the posted review request in a browser window) due to a Python/AppleScript bug. This is Python bug #30392, for those who are interested.

We’ve worked around this. Your default browser will work once again. Thanks to those who pointed this out!

There’s also a whole new macOS installer coming that should actually work on all setups. We’ll have this on the Downloads page once it gets a little more testing.

Git and Git-SVN

Git-SVN users should no longer encounter crashes when trying to post changes for review. That was pretty disruptive.

Git repositories with submodules containing pending changes no longer cause warnings about dirty repositories when posting changes. They’re not included anyway, and just added to the confusion.

Crazy Subversion Diffs

If you had a line of code being deleted that happened to look like a diff header (say, --- XX (YY)), it could cause some code we have for fixing up diffs to get very confused. That, unfortunately, could lead to lines being excluded from the diff, breaking when you try viewing it in the diff viewer.

We’ve rewritten this code to be very careful about these lines. It won’t get confused again.

Team Foundation Server and Visual Studio 2017

Team Foundation Server users who have upgraded to Visual Studio 2017 can once again post changes. TFS has had a nasty habit of changing their file formats, APIs, and command line options, but after much tearing out of the hair, we’ve restored compatibility.

All versions from Visual Studio 2011 onward should work just fine, so no need to upgrade to 2017 just to use this release.

We’ve also fixed a regression when using the Team Explorer Everywhere adapter.

ClearCase and Cross-Platform VOB Lookups

ClearCase users can now name their repositories in Review Board based on a component of a VOB path, instead of naming it based on the entire VOB path. This helps with the differences in how ClearCase represents VOB paths on different platforms. For instance, a VOB path of /vobs/MyVOB or C:\vobs\MyVOB will now match a repository name of MyVOB.

There are also some performance improvements for looking up VOBs.

And Other Such Things

There are improvements to the Python API, such as not prematurely exiting the process, plus compatibility fixes for Review Board 3.0. We’ve also added a new config option to disable certain warnings in RBTools, which would be especially useful for repository hook scripts.

For the complete list of changes, see the release notes.

To upgrade RBTools, visit the downloads page.

Read More

RBCommons and Cloudflare: Don’t worry, be happy!

There was a major security breach announced this week by Cloudflare, a popular service used by millions of sites. This security breach affected customers around the world, causing passwords, API tokens, private conversations, and more to be leaked into search engines and people’s browser sessions.

You probably have a lot of passwords you’ll need to change this week, but don’t worry, RBCommons does not use Cloudflare, nor do the services RBCommons depends on. Your information is safe!

We recommend that you take the time to ensure you’re using strong, unique passwords (ideally stored in a password manager like 1Password or LastPass), and enable two-factor authentication on RBCommons to make your account even more secure.

To learn more about the Cloudflare security breach, and how it affects you, read their disclosure and see the list of sites using Cloudflare to see if you may be at risk.

Read More

RBTools 0.7.7 is released!

We’ve just put out an all-new release of RBTools. Version 0.7.7 features compatibility fixes for various types of repositories, better support for TFS, and some new features to help with common usage and automation.

You can see the release notes for the full list of changes. We’ll go over the highlights here.

Compatibility/bug fixes

In this release, we’ve aimed to fix a handful of compatibility problems that have been reported to us. Thanks to all the contributors who sent patches!

  • RBTools is once again compatible with Mercurial 2.x. This regressed in 0.7.6.
  • Some error displays are fixed when using the version of Python shipped with macOS 10.11.
  • Perforce gained the ability to post against null client roots, and fixed posting ranges of submitted changelists.
  • Repository lookups utilizing mirror paths or Subversion UUIDs now work once again. These regressed in 0.7.6.
  • rbt post for Git now supports --exclude-patterns when using git-svn or git-p4.
  • rbt land no longer crashes if it can’t determine the approval state on a review request.

Improved Team Foundation Server support

The old TFS support was a bit slow, due to the way we had to interact with the Team Foundation Server command line tools. It also presented compatibility problems, as different versions of Visual Studio shipped different, incompatible versions of these tools.

We’ve now introduced new support that doesn’t depend on their tools and is optimized for our use cases. This means better compatibility everywhere, faster posting, and new features.

To start with, we’re adding the ability to post shelved changesets! You can do this by simply running:

rbt post <shelveset-name>

To begin using RBTools 0.7.7 with TFS, you will need to install our new TFS adapter by typing:

rbt install tfs

New features

We’ve added the ability to specify a destination tracking branch for rbt land. To choose something other than the default (say, origin/master on Git), you can now specify:

rbt land --tracking-branch <branch-name>

If you find yourself needing to pass --svn-prompt-password all the time for your Subversion setup, you can set SVN_PROMPT_PASSWORD in your project’s or user’s .reviewboardrc instead. Just set this and you’ll never have to type it again:

SVN_PROMPT_PASSWORD = True

What’s coming next

We’re working toward a RBTools 1.0 release, which will feature enhanced support for Mercurial, new automation commands for use in the upcoming Review Board 3.0, easier setup and installation, and better display of progress when posting changes.

We’re also hard at work on a rewrite of our documentation, with the aim of providing more practical, detailed setup and usage guides for RBTools. These will begin to land over the next month.

If you have any bug reports or feature requests for either RBTools or the documentation, we’d love to hear them! You can file a bug or reach out to us on our reviewboard-dev discussion list.

Read More

The New RBCommons is Live!

We’ve been hard at work these past few months on a major update to RBCommons. This update brings all the many improvements found in the latest version of Review Board.

 

A more refined look

New RBCommons UI

RBCommons has a new improved look. We’ve modernized the look, polishing things here and there, bringing a much fresher feel to the service. Don’t worry, though, you won’t have to relearn anything. We’ve kept everything familiar.

Along with the new look is support for mobile! You can now use RBCommons from the phone, letting you catch up on reviews and new changes while on the go. Mobile diff review isn’t there yet, but is something we hope to bring down the road.

 

Archiving/muting review requests

It’s easier now to stay on top of the review requests that really need your attention. By archiving/muting review requests, you can take control over your dashboard and help you get to Inbox Zero (or maybe Dashboard Zero).

Review requests can be archived, hiding them from the dashboard until there’s new activity. They can also be muted, hiding them completely from the dashboard until you opt into seeing them.

Learn more about archiving and muting.

 

Trivial publishes for review requests and reviews

When you’re making a small change on a review request or clarifying something small on a reply, sometimes you don’t want another e-mail to go out to your team. We’re all busy, and every e-mail we add is one more thing to look at.

RBCommons allows for trivial publishes of review requests and replies. The green draft banner for review requests and replies contains a “Send E-Mail” checkbox, checked by default. To prevent sending an e-mail to your team, just uncheck it before hitting “Publish”.

Learn more about trivial publishing.

Expandable diffs in reviews

Inline Diff Expansion

Ever want to see just a bit more of a diff when reading a review, without having to jump into the diff viewer? Now you can! Just hover over the little snippet of the diff to see the new expansion controls. From there, you can start exploring more of the diff, without ever having to leave the page.

 

Live HD thumbnails for file attachments

Thumbnails now show more of the content you want to see.¬†They’re no longer just tiny previews of a file. Now they’re big and vibrant, and come to life when you hover the mouse over them, scrolling through the file to show you even more.

Learn more about Live HD thumbnails.

 

Revisioned file attachments

RBCommons now tracks every revision of a file you upload. Make a change to a graphic, or a PDF document? Simply update the existing file attachment by hovering over the thumbnail and choosing “Update.” Reviewers will be able to go view any revision, and for some files, they can even diff between them!

 

Diffs for text-based and image-based file attachments

Hey, we were just talking about this!

Image and text file attachments with multiple revisions can now be diffed. You’re seeing one example of this here, with a split diff of two images.

Image diffs make it easy to see how a graphic has changed over the revisions. You can view this in several different modes: Two-Up, Difference, Split, or Onion Skin modes.

Text files can be diffed as well, and this works exactly like the diff viewer.

Working with Markdown? Now only can we diff the source text, but the rendered output as well!

Learn more about diffing file attachments.

 

New review group setting to auto-add new users

Got a review group or two that you’d like everyone to be a part of, automatically? We’ve got a new option for that! Pull up the settings for a review group and toggle “Add new users by default.” Any new user you invite to your team will be automatically added to the group.

 

Browsing and posting Bitbucket commits for review on the New Review Request page

New Review Request

Bitbucket users, rejoice! You can now browse for commits in the New Review Request page. If you work in a “post-commit” model, where you push commits and then post for review, you’ll find your workflow’s just gotten a lot easier.

 

WebHooks for integrating with other services

RBCommons can now talk to third-party services and scripts through WebHooks.

WebHooks are used to notify HTTP services on certain actions (new review requests or updates, new reviews, new replies, etc.). You can use this to interface with in-house tools in response to new diffs or discussions, forwarding them on to other services or automating code reviews.

Learn more about WebHooks.

 

API Tokens for safer authentication

If you’re working with scripts or services that need to talk to Review Board, you can now create API Tokens and hand those out, instead of handing out a password. These are safer, and have the added benefit of letting you limit what can be done in that API session.

Learn more about API Tokens.

 

There’s a lot more, but those are the main feature updates. We hope you’ll like the new RBCommons. We know we’ve been looking forward to using it¬†for a long time now.

If you have any questions or hit any problems, you can reach out to us through the “Need help?” button (bottom-right of any page on RBCommons), or e-mail us at support@beanbaginc.com.

Read More

RBTools 0.7.6 is released!

Today’s all-new release of RBTools 0.7.6 comes with over a dozen improvements, from Mercurial and Perforce fixes to new Team Foundation Server capabilities to automation enhancements.

We’ve fixed some character set compatibility bugs with Team Foundation Server. There’s also new support for posting branched/copied files for review (this requires some changes we’ll be bringing to RBCommons in a big update this quarter), excluding files using --exclude, and specifying a custom path to tf.exe.

Perforce users should see more stability in edge cases, like posting deleted symbolic links for review or when dealing with Unicode mismatches between review requests and changesets.

Mercurial users can now safely use relative, negative, or short revisions when specifying commits to post for review.

We’ve improved RBTools’s behavior when running in a non-interactive console, allowed rbt api-get to be used outside of a source tree, and made it easier to work with paginated responses in the Python API.

Performance has been improved when looking up repositories on ClearCase and Subversion.

These are just some of the improvements made in RBTools 0.7.6. For the complete list, see the release notes.

To upgrade RBTools, visit the downloads page.

Read More

Announcing virtualenv-multiver for Python Development

If you’ve worked with virtualenvs for Python before for development/testing, then you may have hit cases where you really wanted multiple versions of Python installed in your virtualenv. Which, you may actually have working, because virtualenv, in theory, supports this. In fact, you’re supposed to be able to do:

$ virtualenv -p python2.6 my-env
$ virtualenv -p python2.7 my-env

That’d be great, if it always worked. It doesn’t. When your virtualenv gets built,¬†bin/python¬†may end up being a link to¬†bin/python2.7¬†(or what have you), or it may be the contents instead of a link. Subsequent installs may end up overwriting binaries, producing a¬†python2.6¬†and¬†python2.7that are both Python 2.7.

Oh and it gets worse. On Mac, with a standard Python install, these binaries actually end up invoking ../.Python, a symlink pointing to the system Python. This link is not versioned. So much for multiple Python versions in one virtualenv on the Mac.

A solution!

We fixed this. Now you can run a single command to get a working environment going, without messing with things or running into problems on the Mac. This is virtualenv-multiver.

Now, setting up an environment is as simple as:

$ pip install virtualenv-multiver
$ virtualenv-multiver my-env 2.6 2.7

Couldn’t be easier. This works both for new environments and existing ones.

This is a beta, so there may be some issues here or there. If this is useful to you, give it a try and let us know!

Read More

RBTools 0.7.5 is here!

RBTools 0.7.5 is now out and ready to install.

This is largely a bug fix release, focusing in part on improved compatibility with Windows, Git, Subversion, Mercurial, Perforce, and Team Foundation Server.

On Windows, RBTools will now first look in %HOME% to find any custom .reviewboardrc files, instead of only looking in the Application Data directory, which will be quite helpful with many system configurations. There are also fixes for using Mercurial on Windows.

Non-Git user? You’ve probably seen that annoying but harmless command not found: git error when posting a change. That’s gone now!

For Perforce users, posting submitted changes or files outside of the client view now work. This had regressed in an earlier release, but you should be in good shape now.

Subversion has seen some more Unicode fixes, plus fixes for rbt post --svn-show-copies-as-adds.

Along with all this, we’ve added a new feature for setting a custom search path for .reviewboardrc. You can set your $RBTOOLS_CONFIG_PATH to a list of paths to search, allowing you to make your version in $HOME take precedence over what’s in your repository, and allowing you to work with centralized collections of aliases in your organization.

See the release notes for the complete list of changes.

One more thing: We’ve simplified installation for those of you using pip to install. Our builds are now directly hosted on PyPI, meaning all you now need to do to upgrade is run pip install -U RBTools.

Read More

Introducing new special user permissions

As a team grows, it often becomes the case that more developers need to assume more specialized roles in the code review process. Not just that of developer and reviewer, but also that of a manager of sorts, helping to keep the review process going and to keep the process tidy.

We’re introducing a few new special user permissions, designed to give users a subset of an administrator’s abilities. These can all be set in the Team Administration page by clicking the pencil icon next to team member.

 

 

The first permission, “Can close or reopen review requests from other users,” enables a user to help keep the list of review requests tidy by toggling whether a review request is currently open. If you’re not auto-closing review requests, if you have review requests open from former team members, or if you’re managing an open source project on RBCommons, this can be quite handy.

The second permission, “Can edit review requests from other users,” allows a user to modify a review request on someone else’s behalf. They can upload diffs, edit fields, and so on. The changes currently appear as if they’re from the owner of the review request.

The final permission, “Can post review requests as other users,” is most useful for scripts. In cooperation with RBTools (using rbt post –submit-as), a script can post a review request on another user’s behalf, perhaps when a change is committed to a special branch, or after a sandbox operation passes.

We’ve been piloting these permissions with some projects for a while now. Please let us know how they work for you, and if you have any questions or problems.

Read More