Quality testing, program checking, online QA... whatever you call it, it has become a very important part of the online survey creation process. Last night, Bieke and I got a chance to attend a presentation on quality testing by Nathen Garcia at Enlaso in Boulder. He has years of experience quality testing many different types of projects, from video games to mobile devices, and it was interesting to hear his perspective.
For us, program checking generally consists of running through the online survey created from a questionnaire we translated, and making sure that the programming of the survey did not create any new linguistic issues. It is something I started doing back in 2005 on English language surveys we created at LRW. There are new challenges that come with program checking in many languages like we are doing today.
The biggest, most important word that I heard last night was the first topic of the night: scope. I have no doubt that our clients always know what they are expecting when they send us an in-language survey for program checking. However, different clients actually have quite different expectations, and it is important for us to understand exactly what each client wants. Nathen's example last night was when a client asks, "how long will it take to test this program?" His answer is, "how long do you have?" because he could do it in 5 minutes or 100 hours, depending on the client's needs. That is very true: we can spend as long as someone wants us to looking for errors. So the scope boils down to: 1) what specifically do you want us to look for, and 2) how long do you want us to spend on it?
The other issue, which is something that has slowed down some translators in the past, is the test path. As market researchers (myself included), we tend to just assume that our program checkers will have no problem reading the programming instructions to keep from getting terminated and to make sure they check all screens. In reality, even though the linguists can usually decipher the (sometimes very complicated) programming language, figuring out programming instructions can take up a lot of time that they could be testing for errors. And it also creates a much greater chance that they miss something important. In an effort to avoid this, we are going to be focusing much more on creating an accurate test path for the linguists up front, especially in multi-language projects.
Overall, this event was a great experience for us. Nathen broke out the magnifying glass and looked in great detail at some of the biggest issues we face in program checking. Thanks to the CTA for organizing another great professional development event!
Best,
Geoff
