Effective Exploratory Testing (Part 4): A Practice of Pair Exploratory Testing

Part 1: Effective Exploratory Testing (Part 1): Some good practices
Part 2: Effective Exploratory Testing (Part 2): More Effective With Pair Exploratory Testing
Part 3: Effective Exploratory Testing (Part 3): Explore It, Automate It
Part 4: Effective Exploratory Testing (Part 4): A Practice of Pair Exploratory Testing (Reading)
Part 5: Effective Exploratory Testing (Part 5): Risk-based Exploratory Testing
Special Series 1: Effective Exploratory Testing: Empower Your Exploratory Testing With Smart Tools (1)
Special Series 2: Effective Exploratory Testing: Empower Your Exploratory Testing With Smart Tools – A Case Study (2)
Context-Driven Testing Framework: Effective Exploratory Testing: Context-Driven Testing Framework

 

Usually, we see a baby’s mind as a blank slate, we believe that s/he is irrational, egocentric and unable to think logically. However, recent studies have proven that babies and young kids understand, observe and explore, imagine the things surround them better than we think. Especially, in some ways, they learn new things faster and smarter than adults that we can consider them to make our exploratory testing better, especially it becomes more effective with game testing

Here is a story from my baby: Everyday, I usually go home a bit early to play with my daughter (her name is Aurora), I would spend my time for the family as a way to balance between work and life. My daughter is very young (she just turned into 1 year old).  Last month, I just bought a new glass (the new one was rather similar to the one I was using) on the way I going to work. After a few hours in work, I totally forgot about my new glass. When I went back home, Aurora was staring at me for a while. Then, she started crying. My father, my mother and my wife were there, all and including me could not understand why she was crying. I tried to think what I have acted to her. A few minutes later, I just remembered my new glass and understood the reason. At this point, it turned out a couple of questions:

• Why could she recognize a new change from my face very fast?
• Why my father and my mother who have been growing me up and watching and seeing me for many years could not realize this change while she easily caught it? Even my wife could not know it too. Or they knew it but they could not see that is the cause of the baby crying.
• If we can find out how kids learn new things, is it help our exploratory testing? ^[1]

^[1] Exploratory Testing is an approach to software testing that is concisely described as simultaneous learning, test design, and test execution. Exploratory testing is more effective if we can learn the product faster, build as many as models and recognize the problems during our learning.

By the prefrontal cortex development, adults experience functional fixedness that makes adults see everything exactly as it is. While kids have their brains designed for learning, so they look at an item and building lots of images of it. When we grow up, we have had too many images and experience on what we are facing, so we only see the things in our patterns which are built by our experience. Even in the way we learn the application is still different from children. My daughter observed how my father was operating his iPad, when the iPad handed to her. She repeated his actions but did not hesitate to touch to anything appearing at her eyes. While we usually use an application in the way we have known it or was trained. These have negative impacts on our testing. For instance, if we test an application many times, we may miss new context created by its changes. Or if we have tested “HR System”, we may easily fall into pre-defined context and hard to realize new scenario from new HR System.

We are not a baby, so we cannot do like that. And more importantly, the baby could not perform the testing when they are not disciplined and cannot remember what they have just done. However, take into account this point would help us shape our testing better. At MeU Solutions, sometimes we reshape the exploratory testing that we call “Exploratory Testing in Fresh Eyes”. This testing is applied for charter that have been run many times. The testing is in the form of pair-testing. It is a pair of experience and new in term of understanding and acknowledging about the product. There are two testers who have similar testing experience level. But one (primary) has much experience of the module (or application) being tested, he is a person that has run the charter. And another (observer) is very new, he even can be a tester from other projects.

The following are steps taken for “Exploratory Testing in Fresh Eyes”

• Observer to run once the charter with previous version of module (previous build). The purpose is to have first images about the module
• Primary and Observer to perform pair-test in a session with the current version of that module (current build)
• During this session, observer like a baby to perform a range of activities as follows:
  • watch all actions of primary, take notes questions
  • take notes any difference between this version and previous version
  • take note new test ideas
• When the session completed, both to discuss on what have been noted and tested. Turning out difference between two

“Exploratory Testing in Fresh Eyes” takes more time of testing, but it benefits on some aspects

• Like a baby, observer doesn’t have much experience on the module or having many images of module. He is much easier to see changes or abnormalities in two versions
• With fresh eyes, fresh mind, observer is more creatively, so he has more test ideas
• He may drive the software into the scenario that documented or acknowledged before

We have acknowledged great benefits of this approach from experiments used in game testing. The game is a kind of adventure. Exploratory testing is one approach is used popularly in almost game testing at our company, although there are some other methods such as: Effect (Background Music and Sound) testing, Functionality (Scenario-based) Testing, Performance Testing, User Interface… A few charters have been chosen for the experiment. The following are our achievement highlights:

• Faster to recognize minor changes. At a stage in a game when the character has to dive the ocean. The color of ocean was not changed according to the level of ocean deepness if compared with previous versions. This issue has been identified by the observer
• Fresh Eyes look everywhere on the screen instead of staring at the character: Movements of octopus were not natural
• Don’t follow the script and the instruction: Crash occurred when the observer played the game in a wrong order and skipped some steps
• New experience causes new damages: Because the observer is new with the game, he failed at some stages for many time that made him angry. He touched and pressed buttons rapidly at different places that made the game shutdown

A practice can be good for this context, but it may not work with another context. “Exploratory Testing in Fresh Eyes” is not methodology, it is just a practice. Take time to experience with it can benefit for your testing.

If you have any question, I like to assist you with this approach. Please feel free to contact MeU Solutions.

By Flowerpecker (a Tester at MeU Solutions)