PCF Testing

We have gotten to the point where we can use a lot more volunteers to help us with the Miro 2.0 nightly build testing.  If you haven’t run any formal regression tests in the past – or are completely new to the testing team, here are some steps to get you started.

1. Go to the Litmus login page and create a new account – or sign in with your already existing one.
   

An irc nickname isn’t required however it will be be the preferred identifier in results reporting and can help prevent your email or real name from being published.

2. Login to litmus and select a test run from the list of Active Test Runs.
   The recommended test run is marked with a yellow star.
3. Specify your testing configuration.  Toggle the radio button that matches your platform, select the OS from the pulldown menu, specify your locale setting and click submit configuration.  If you don’t see an option relevant to your configuration, please contact me directly (jed at pculture dot org) or comment here.
   
4. Choose your testgroup and subgroup.  For new testers, it is recommended that you start with the ‘Quicktest’ group and ‘The Basics’ subgroup.
   
5. Follow the instructions and mark each test Pass or Fail.

Here are a few extra helpful hints:

• Use the Show All button on the right to expand the text of all the test cases.
• If you think you found a bug, click the View button to the right of a test case.  It will show you all reported results for that test case and list any bugs associated.

If you encounter a new bug, please report it. The Miro Help menu has a Report a Bug option.  Once you have created the bug, please Include the bug # in the test case results and add a brief comment about your finding.

It feels like every aspect of my digital life has recently become cleaner, clearer, fresher, better.

I just updated to Firefox 3 RC3 today.  I am so psyched to be part of their world record setting download day.  I’ve also been running Ubuntu Hardy for a few weeks now, and of course I have been testing our nightly builds of Miro.

Each project just gets better with each release.

Sometimes it hard to tell  how far you’ve come, when each change comes in small increments as opposed to big jumps.  So the other day – I downloaded Democracy 0.8.5.3 and installed it.  That was the latest version when I first joined up with PCF almost two years ago.

 Democracy Player 0.8.5.3

Passing through a plethora of database, ui, torrent client and preference changes, the update to the latest nightly was flawless.   Here is a sneak peak of what we have been working on.

2008/06/13-nightly build:  Have I ever mentioned my huge love for the dev team at PCF?

So this is where we are right now.  Want to know what’s in there?  There is a draft copy of the release notes for 1.5 that outlines the changes and bug fixes in the current branch.

We’d love to know how it works for you – so backup up your database, and help us fine tune the next level of internet video.

I think many people are hesitant to jump into testing because everyone knows that the nightly builds are  by definition, unstable, and people already have their channels set up just as they like them.

There are two ways you can preserve and restore your data prior to starting a test run, OPML export, or by copying off your database.

Export Channels OPML

This option is built into miro and can be accomplished as easily as going to the Channels menu and selecting Export Channels (OPML)… . This will create a list of your channels and channel folders that you save off to your system. You can then import the list at any time using the Channels menu and selecting Import Channels (OPML)… .

Using this, will only save off your channel list. If you delete your database, and start over with a clean system, Importing the channels via OPML will not preserve the downloaded, watched or unwatched status of videos in each channel.

Database Backup and Restore

Currently database backup is still a manual process and the location varies with your operating system.

• Windows XP: C:\\Documents and Settings\(you)\Application Data\Participatory Culture Foundation \Miro\Support\sqlitedb
• Vista: C:\\Users\(you)\AppData\Roaming\Participatory Culture Foundation\Miro\Support\sqlitedb
• OS X: /Users/(you)/Library/Application Support/Miro/sqlitedb
• GTKX11: ~yourhome/.miro/sqlitedb

Just copy your sqlitedb file to a safe location on your system.  If something goes wrong during testing – you can do a fresh install, copy your database back to the original location, restart Miro and be back up and running.

* Note:  If you migrated your movie storage – restoring the old db would cause it to lose track of those items.

I hope these two options make you feel a little more comfortable about diving in to some of the testing builds.  If anyone out there wants to help make the backup / restore process a little less manual – reply to this post.

Leading up to the 1.2.3 release, I was really happy to see that some new people signed up and helped us with the regression testing.

New testers like jan.s,  fixitguy, fox.box, tbutterfoss and strash joined some of our regular testers like i_therain, sedatg and elmargol to help get this release out.  I also received some emails from a few people who didn’t get a chance this time, but would still like to in the future.

Right now is a really exciting time to be part of the Miro QA team.  As a Webware 100 winner, it is official:  We ARE are one of the coolest apps in town.   If you would like to be part of the Miro QA team you should do these three things:

1. Go to the QA wiki to browse our to do list.
2. Sign up for the tester mailing list to stay up to date on our activities.
3. Join us for a global chat on the Miro chat channel  (or join irc.freenode.org #miro) this Friday at 7:00 PM GMT.  This is 9:00 PM CEST, 8:00 PM BST,  3:00 PM EST, 12:00 PM PST.

I can’t really buy anyone a beer from here, but you can BYO.  I’d love to meet you, answer any questions you might have and share ideas about how we can continue to improve Miro QA.

I’d like to believe that we did the update releases for 1.2 just so that we could say that a few times… but we didn’t.

As I explained in an email to the mailing list, we really need to get more people involved in the testing so that we get more systems variation.

So I am hoping that some up you can help out by picking up the rc build, for your respective os and run the Quicktest for 1.2.3 rc0.

If you have time to help – drop a line here, send me an email jed [at] pculture [dot] org, or just find me on the irc (irc.freenode.org #miro).

ok – so i lied -

We need some help testing the Miro 1.2 release candidate builds.

Last night Will posted the rc0 builds for the upcoming 1.2 release. There are some interesting bug fixes and updates included in this release as well as some serious behind the scenes changes.

Most interesting are some of the enhancements we have made to the preferences.

1. We have added the highly-requested feature to allow a global option to control channel autodownload settings.
2. Separate preferences to control manual and automatic simultaneous downloads.
3. The ability on linux to toggle between the xine and gstreamer backends.
4. A preference to set the torrent upload ratio.

There were also some big changes to fundamental parts of the application. They shouldn’t have any obvious effects but definitely require thorough testing prior to release. This includes:

1. Frontend – Backend reorganization (devs can check out Ben’s blog on this)
2. Update to VLC 0.8.6e (affecting windows)
3. Update to Perian 1.1 (affecting os x)
4. Update to XULRunner 1.9 (affecting windows)

The release notes draft contains a more complete summary of the changes.

Please lend a hand

As usual the builds are located on the nightlies page.

The tests are located in the litmus regression testing.

For more ways to help – please check out the QA wiki.

In the works, we have another short-cycle planned with a few key fixes. You can expect a call for help with the rc testing in a week or two. However, there are also some major changes going in the development background. Most recently Ben has done a bunch of frontend / backend separation work.

When I dive into the latest nightly build testing, I take a pretty mechanical approach. Does it work? Download? Search? After that I start working on the regression testing either the litmus tests or the Test Cases from the wiki page. Repetitious at times, but I am the QA and it’s my job to get to those hard to reach places.

One of our volunteers, sedatg, mainly a mac user, is also a pretty reliable nightly build tester. However, he has a different approach. He is a truly engaged user and seemingly a vapid consumer of online video. He also seems to suffer from bandwidth limitation.

So his focus when testing is about his experience using Miro to download and watch videos. He uses the irc (irc.freenode.org #miro) to communicate and troubleshoot issues with the developers and other users. He is also active in bugzilla to vote on the tickets that most concern him and communicate ideas and solutions. He claims it’s all self-interest, but whatever, it makes Miro better for all.

Another of our volunteer testers, fluteman also appears download a lot of video. I have imported his OPML channel list, and it is long and well organized. He is more of a conservative tester, usually waiting for a release candidate before moving beyond the stable release. But because of his huge channel list, he regularly manages to push Miro to it’s limits. He finds bugs that maybe not everyone would experience which keeps us on our toes and lets us know what’s lurking.

Over the next few months – we will be implementing changes to improve performance and presentation. And testing will be all about regression, regression, regression.

Biodiversity can be used to measure the health of biological systems. So whether you are a dragojevic, a sedatg, a fluteman or some other unidentified species type. Your unique approach can help maintain the health of Miro.

Please use the links in the previous post to get the latest release candidate for testing. Or you can just go to the Nightlies page.

Will has posted RC1 and this is what he has to say:

Differences between rc0 and rc1: r5953 - fixed #9414: adds the r number in the About box on GTKX11

r5957 - some OSX fixes

r5961 - fixed #9375: fixed keyboard shortcut so cmd-D stops video in fullscreen r5963, r5968, r5969 - NullSoft installer changes to install iHeartMiro extension

All Miro 1.1 rc1 files are in the sticky section on the nightlies page.

It'd be cool if we could get some serious testing with rc1, identify

 issues quickly, and then fix fix fix...

So please grab the build and check it out. If you can take the time to run some our litmus tests it will give a good sense of how the build is looking.

We are ready for Miro 1.1 RC testing to begin. You can run the tests in litmus – I recommend the 1.1 pre-Release testing, test run. Start with the Quicktest then move to the New Feature and Bug Fixes section.

• Windows 1.1 rc
• OS X 1.1 rc
• Source 1.1 rc

The biggest change for 1.1 relates to torrent downloads. We have switched over to using libtorrent, and have implemented some long asked for options. You can now control both the upload and the download rates or torrents. We also display the total number of seeders and leechers.

Other changes include updates to the navigation buttons on the Guide so that the forward, back and home will work for any additional guides added to Miro. We have also implemented a combined search feature.

You can check out the weekly testing page for a more complete summary of changes.

If anyone wants to help out with the creation of more tests, please let me know. You can reply here.

After some build troubles resulting from the change to libtorrent and a few other things, the nightly builds are back and better than ever.

We have changed over to using libtorrent for the torrent downloads. Along with this change we have updated the torrent preferences to allow users to configure the bandwidth rates for upload and download, forward ports and ignore unencrypted ports. We have also expanded the details to display the total numbers of seeders and leechers.

The testing needed for these changes is documented on the the Weekly Testing wiki page.

If you want to take a look and help expand some of the testing – I marked a section where I could use some additional help with that.

We would also love to know if it seems faster to you?