Automating Litmus tests at Yahoo!

Tyler Hall wrote to tell us how his team at Yahoo! is using Litmus as part of their deployment process. I think it’s a fascinating example of how to use Litmus. Note that it is fairly technical and involves some manual configuration, though.

yahoo-code-sample.pngTyler’s PHP script (which is available on GitHub) automates the process of starting a browser test on Litmus. He’s set this up as a deployment task in Capistrano, which they’re using to deploy their site to their staging and production environments.

Each time they deploy a new version of their site, their script triggers a Litmus test of their pages, using the A-grade browsers recommended by Yahoo.

He explains that these automated tests are useful for a number of reasons…

It starts the test faster, so by the time we get around to checking the results, they’re complete or nearly done.

Perhaps more importantly, this provides a visual history of our changes. All of our code is in SVN, of course, but SVN doesn’t help when your boss asks “when did that background color change?” or “are you sure the site was working on April 23rd?”. Having a log of what our sites looked like for each revision/push is invaluable.

We use SVN and Capistrano at Litmus to deploy our site, so I’m going to use this script to save myself time when we make design changes.

My thanks to Tyler for sharing the script and his process for testing at Yahoo. If anyone else has any interesting ways they’re using Litmus please email me or leave a comment here. I’d love to hear about them.

Subscribe to our newsletter

Get all our best stuff in your inbox, just a few times a month. We'll never share your email address and you can opt out at any time.

  • Carlton

    Awesome, thanks for sharing Tyler and the Litmus team! Having a play with it now and can definitely see us using it.
    I have a small question though, I notice the Yahoo guys are using the default browser sets for which tests to run. I would like to be able to allow the user to select the browsers they would like to test in before kick starting the script.
    Are the IDs for the checkbox input element representing browser choice likely to change in the near future because they seem pretty randomly numbered at moment?
    ie.
    test_set[checked_client_ids][2] -> IE 6 (grrr)
    test_set[checked_client_ids][94] -> Flock 2.0

  • http://litmusapp.com/ Paul Farnell

    Carlton: That’s great to hear! No, those IDs shouldn’t change at all – they’re the IDs we use internally. However, over time the line up of browsers will change, and eventually we may drop some of the oldest ones, so just keep an eye on the list. We’ll announce any changes here on the blog, too.