Effective Exploratory Testing (Part 2): More Effective With Pair Exploratory Testing

Part 1: Effective Exploratory Testing (Part 1): Some good practices
Part 2: Effective Exploratory Testing (Part 2): More Effective With Pair Exploratory Testing (Reading)
Part 3: Effective Exploratory Testing (Part 3): Explore It, Automate It
Part 4: Effective Exploratory Testing (Part 4): A Practice of Pair Exploratory Testing
Part 5: Effective Exploratory Testing (Part 5): Risk-based Exploratory Testing
Special Series 1: Effective Exploratory Testing: Empower Your Exploratory Testing With Smart Tools (1)
Special Series 2: Effective Exploratory Testing: Empower Your Exploratory Testing With Smart Tools – A Case Study (2)
Context-Driven Testing Framework: Effective Exploratory Testing: Context-Driven Testing Framework

 

Pair Testing is an approach of software testing in which two persons sit together at the same time and place to carry out and design tests. Pair testing is a collaborative effort, versus a single-person effort. Usually, one person controls the mouse and keyboard to perform directly the tests, while the other person asks questions, takes notes, and brings more scenarios which support the testing better. When talking about pair testing, we usually imply that one tester and one developer to work together.

 

Session-based testing is a method to measure and manage & implement exploratory testing better. In session-based testing, the testing is limited in given timeboxes, typically 60’-120’ minutes long. Each test session has its specific goals and contains a specific portion of what needs to be tested. A session can be a UAT Test for a story, a screen, a bug, a business transaction, or a technology facing for a piece of feature

In a session, tester will pay attention on something to exploit an (or many) objects. Although session-based testing comes up with many benefits, it still has disadvantages such as:

• Inattentional Blindness: an effect where paying too much attention to one thing can result in not seeing something else significant, even if it is quite salient.
• Problems with taking Notes: The final step in session-based testing is a debriefing session. A debrief is a discussion around a recently completed exploratory testing session between two people – one is tester who has just done the session, and another is who with much more experiences. A debrief session is only effective and useful when the tester has been taking his notes at certain level of details. However, it is not much easy to do that. During executing the tests & exploring the application, almost testers are fallen into their testing flows & logic and forget to take necessary notes (or their notes are not informative enough) to debrief later on.

From our practices, a combination of session-based testing and pair exploratory testing would help solve these problems and also to generate more additional test ideas along the way.

The following are kinds of pairs in exploratory testing. Each pair associates with different benefits and its purposes

 One Exploratory Tester versus One Exploratory Tester: Both of two testers should have the same the product knowledge (or at least with the charter they are carrying out). One tester plays the primary role to perform the testing while another observes & takes notes for all activities and what is happening in the application, abnormalities are identified by this second tester. The purpose of this combination is to improve the exploratory testing over time as well as suggesting more test ideas on the way of testing. Certainly, this approach will compose notes in detail which are much more useful to use for debriefing

One Exploratory Tester versus One Product Owner (or business analyst): This combination is very effective in generating test ideas and designing the tests which touch almost the corners of product. The testing skills and critical thinking of exploratory tester combine with wide product knowledge from Product Owner (Or Business Analyst) will deliver the productive testing in both depth and breadth. The power of product knowledge also helps recognize problems faster and build hypothesis accurately. However, it is not easy to ask the Product Owner to spare his time for pair-testing, so it should only occur in important sessions.

One Exploratory Tester versus One Automation Tester: When you don’t just want to improve your product exploration, but you need to make your session reusable. Firstly, I would prefer think of test automation in terms of what we say the Context-Driven Testing: test (or check) automation is any use of tools to support testing. While one tester does exploration (investigation) on a product, he does whatever comes to his mind by thinking in reverse direction as how this piece of functionality would break, automation tester thinks of what can be automated to increase the effectiveness for next session. The outcome of this tester is a set of screens, flows, and actions which is candidates for automation. At the end of the session, these two testers to take 15’ to design the test results and formalize the set of automation. I will have an article called “Effective Exploratory Testing (Part 3): Explore It, Automate It” to explain this approach

One Exploratory Tester versus One Developer: I heard this approach used by many organizations. The benefit is to have many diverse views on the product.

 

Every pairs have its benefits. It depends on what our goal and more important is to avoid personality clash.

Following steps to perform Pair Exploratory Testing with a Session

• Determine The Goals: Answer the questions such as: Why need to perform pair exploratory testing with this session? What outcomes? Who should participate (automation tester, PO, BA or Developer, another exploratory tester)?
• Prepare the Environment: It’s important to be prepared and have the appropriate environment setup. The room should be comfortable and quite enough for testing and discuss. Tools need be setup such as: automation tools, video recorder…
• Pair with Who: Determine who can pair with. Personality, Skills, Product Knowledge are selection criteria
• Determine The Scope of Each Person: Identify outputs when the sessions completed, the scope of work
• Testing and Take Notes: Use the testing time to discuss ideas, scenarios, and to try different things. When the session is over, take the time to log well-written bug reports. At MeU Solutions, we use One2Explore tool to keep track the testing records.