On April 19th at 10am Eastern Time, the day and time had finally arrived for the NRG Global Test Competition. Matt Heusser posted the competition rules and off we went. A few weeks later, after a good amount of time and effort spent judging, comparing reports, discussion and chat, the results of the competition were posted (you can read them here). This was the first time I was working with Matt as well as the other volunteers including Jason Coutu, Smita Mishra, and Lalit Bhamare among others. I spent a good amount of time being involved with the test competition and every minute of it was worth it. First and foremost because I had fun and furthermore, I learned a great deal working with the other volunteers in setting up the test competition and from the test competition itself.
Matt first started floating the idea of organizing a test competition on twitter in early January. We had our first meeting via Google Hangouts in mid January. Some of our meetings were held in the evening (Eastern Time) which worked out well for me as my mind was more than warmed up and flowing with ideas and thoughts after a day at the office. Other times our meetings were held in the mornings (7am Eastern Time) as some of our teammates are in India Standard Time – it was much more challenging to get the mind warmed up and flowing with ideas before that morning coffee 🙂
It was a great learning experience being involved as a volunteer & test judge for the competition. We discussed what we wanted to do – and then potential solutions (the how to part). For example how we would communicate with the participants during the competition to answer questions? Where would participants log bugs? What did we want to consider when grading and how would we use our grading scale. Working with the team in determining all of this was great, I learned a lot from it and have a good range of knowledge to perhaps organize a similar event at the office – and have the team as external test judges (which would be awesome). I also had an opportunity to use and familiarize myself with Telerik TeamPulse in the weeks leading up to the the test competition – this was the tool used by the participants to log bug reports.
For the competition itself, we had 17 teams registered from four different continents. Teams varied in terms of number of team members and the location of members within the teams. Teams were given 3 hours for the functional portion of the competition and had scheduled time-slots over the course of the weekend for the performance portion of the competition. Teams had 4 different websites to choose from to test. Some teams choose to execute tests on all 4 websites while other teams chose to perform testing on a select 1, 2, or 3 of the 4 choices. Now the challenge here wasn’t just to read the rules, ask questions, select websites to test, coordinate with team members, implement test strategy, do the actual testing, log bug reports, write test reports – it was doing all of this (and possibly even more for some teams) in 3 hours! This is an actual and real challenge we as software testers face every day – we don’t have all the time in the world (and often very little time) to test so we have to choose what we test and how, wisely.
I reviewed every bug report and test report that was submitted at least twice, comparing the bug reports logged and the content in those reports to our grading scale, and to bug reports logged by other teams. I reviewed how well and clearly the test reports were written, and how valuable the information in the test reports were for me viewing as a stakeholder. I went into the websites and tried to reproduce a lot of the bugs that were logged according to the repro steps provided. I was very impressed with some of the bug reports and test reports we received and that those teams were able to produce and submit this information in a timeframe of 3 hours. I even got a few ideas from one or two test reports that I may be able to apply to certain applications I write test reports for.
Having fun and learning – for me being involved in the test competition as a volunteer & test judge, these two factors went hand in hand. As Matt mentions in the Test Competition Results post, it’s rest and regroup stage for the time being, but I am looking forward to what comes next!
Congrats to all the winners – well done!