PCF Testing

After some build troubles resulting from the change to libtorrent and a few other things, the nightly builds are back and better than ever.

We have changed over to using libtorrent for the torrent downloads. Along with this change we have updated the torrent preferences to allow users to configure the bandwidth rates for upload and download, forward ports and ignore unencrypted ports. We have also expanded the details to display the total numbers of seeders and leechers.

The testing needed for these changes is documented on the the Weekly Testing wiki page.

If you want to take a look and help expand some of the testing – I marked a section where I could use some additional help with that.

We would also love to know if it seems faster to you?

First off – I want to give a huge Thank You to everyone who took part in the rc0 testing. This is the best response we have ever had in terms of the number of times the build was downloaded and results reported.

Within the past week there have been a number of bug fixes. Release candidate 2 has been created and a new test run has been posted on litmus.pculture.org.

This run includes the Quicktest that you ran before, as well as another group called New Features and Bug Fixes. A few of these tests are a little more advanced and cover some of the functional areas that we have been improving.

Please Download 0.9.9.9-rc1 and select the 0.9.9.9-rc1 test run from litmus.

• Source: Miro-0.9.9.9-rc1.tar.gz
• Mac: Miro-0.9.9.9-rc1.dmg
• Windows: Miro-0.9.9.9-rc1.exe
• Litmus: 0.9.9.9-rc1 testrun

Thanks for all your help!

This week Matt put a lot of effort into getting Litmus up and running for us. Litmus is a test case management / QA tool created by the people at Mozilla. We have chosen to start using it to improve the presentation of the regression testing and results reporting.

You can check it out at litmus.pculture.org.

Currently there is a nightly test run configured and a testgroup called Quicktest. It is designed to be a very fast, easy way to check out a nightly build. Within a day or so, I will have the tests from the weekly testing page added into a group called New Features and Bug fixes.

Other than that – here is a quick list of the changes implemented this week that could use your attention. For any of this testing, I would recommend using the 20071015-nightly build for windows or mac or linux souce.

• Channels remember their searches
• Size and Date is displayed for watched folder items
• Only migrate items that were downloaded by Miro
• Change the first time startup dialog
• Changes to the video information displayed in the playback window

The full list and recommend test cases are posted on the weekly testing wiki page.

The team here at PCF recognizes the importance of testing and the value that it adds to a project. As I continually ask friends, family members and sometimes complete strangers to help out with the testing – I often suspect that people are a bit fearful about diving in.

“I’m not a tester”, “I don’t know how to write code”, “I never tested anything before.” These are some of the phrases that I regularly hear, and once and for all – I just want to say – It doesn’t matter.

In my opinion, anyone can be a tester. All you have to do is try a few things, maybe follow some instructions and report the results.

In order to be a Good tester, there are a few more standards, but perhaps not what you would expect

• Education… is completely irrelevant.
• Programming skills… we leave that up to the developers. One of our biggest concerns is the end user experience. If you, as a non-programmer, have trouble using or understanding Miro, that is a bug. It is preferred that you communicate your finding in a sentence or two.
• Courage… You have to be willing to try. If you find something that you think is a bug, report it or ask someone. Don’t be afraid to be wrong.
• Creativity… Look at what a particular feature does – then think about what else it might do, or could do, or should do. Then think of ways to make it do things that it shouldn’t do. Then, write it down, so someone else can expand on your ideas.
• Diligence… Testing isn’t always fun, and can sometimes be a little repetitious. But those regressions tests need to be run. The more people we have looking at the builds on a regular basis – the better Miro will be for everyone!

So – if you are ready to join us, sign up for the mailing list, or subscribe to this post, and say hello on the irc.

We have a new addition to our testing builds page. Starting today, we are regularly posting the source for any and all of our dedicated linux testers.

Here is a link to the build docs for any extra info you might need.

Weeks end: Summary for October 1, 2007

Note for anyone who has noticed the date above: it seems slightly more practical to me that the testing week concludes on a Monday rather than begins. So in that spirit, I will use this space to summarize the changes that have been implemented between Mondays (even though I am well aware that today is Tuesday).

The most significant addition is the implementation of thumbnail generation and length extraction. For Windows this was tried before but resulted in crashes and unstable behavior and had to be pulled from prior releases.

The developers have changed their approach and now use a separate process. The change is also being implemented for linux and os x. If you are interested in the technical information, you can read about it in the MovieDataProcess wiki.

If you want to help test out this feature, there are some test cases in the new Weekly Testing wiki page.

Weekly Releases

The new Weekly Testing wiki page is meant to complement this blog. It contains more details about the latest changes to the release, suggested test cases and known issues. I encourage anyone who wants to help to take the recommended build from the nightlies page and work through the tests.

Additionally – you can edit the page with additional tests you have run or believe should be run. Once things have been wrung out – I will use the information there to update the official test cases for Miro.

Your involvement at this stage of the process will really help prevent defects and provide necessary input to the development team.

Every Tuesday at 21:30 GMT (5:30 pm EST, 2:30 pm Pacific) we have the Team Miro meetings in the miro chatroom (irc.freenode.net@#team.miro) which are open to whomever would like to join. It is a general discussion of volunteer activities and a place to kick around ideas and make them happen.

Last weeks discussion partly focused on the Miro testing community and how we can attract more volunteers to help test the development builds. We need to grow our base of regular testers so that we can increase coverage of operating systems, video drivers and system configurations. We also need people who can test regularly using non-English language settings.

One of the ideas from the meeting was to create this blog. The goal is to provide a weekly summary of the bug fixes, implemented features and known issues in the testing builds. We will also suggest areas for focused testing.

We are also currently working on other ways to improve the testing processes to make test execution and results reporting more user-friendly. I will write more about those later when they are more developed. Until then, here are the current testing builds and documents.

We work on this project because open media matters. Please join us here and in the chat room. We need your help and ideas to ensure that Miro is the choice of video creators and viewers.