Effective Exploratory Testing (Part 3): Explore It, Automate It

Part 1: Effective Exploratory Testing (Part 1): Some good practices
Part 2: Effective Exploratory Testing (Part 2): More Effective With Pair Exploratory Testing
Part 3: Effective Exploratory Testing (Part 3): Explore It, Automate It (Reading)
Part 4: Effective Exploratory Testing (Part 4): A Practice of Pair Exploratory Testing
Part 5: Effective Exploratory Testing (Part 5): Risk-based Exploratory Testing
Special Series 1: Effective Exploratory Testing: Empower Your Exploratory Testing With Smart Tools (1)
Special Series 2: Effective Exploratory Testing: Empower Your Exploratory Testing With Smart Tools – A Case Study (2)
Context-Driven Testing Framework: Effective Exploratory Testing: Context-Driven Testing Framework

 

Exploratory testing is the simultaneous learning, design, and execution throughout all heuristics and test-oracles. It emphasizes the freedom, responsibility and the wisdom (skills, knowledge, and care) to optimize the work. There are two flavors of exploratory

• Known-Unknown: we aware that we don’t know (or don’t know much) of it. We need to explore to uncover risks that threaten the value of the product.
• Unknown-Unknown: is all the stuff that we don’t know that we don’t know it. This needs time, care and skills to recognize.

Obviously, according to this definition, it is something that we cannot automate. Yes – we cannot automate our learning, we cannot automate our responsibility, we cannot automate what we look straight and are curious about it. Except when we can invent a machine which can replace people to care all life facets like the perspective of the famous movie called “The Terminator”

The automation here should be understood as the use of tools to support testing. It is nonsense to use automation for repeating the same exploratory tests continuously across that violates the spirit of exploration. However, it would be reasonable to make the test session (*) reusable in checking whether incremental application changes break your existing functionality. Doing that, then, you will release your time to lead it into other angles to chart the uncharted territory of an application under test, take more time on designing and performing experiments for problems to make sapient decisions and more important is to spend time on uncover Unknown-Unknown world. Exploratory Testing helps us to discover the stuff that we are not aware of, when we have that awareness, we could have written automation checks for them.

However, the opposite of two sides of a coin (between testing vs checking) is a great obstacle to prevent automation applied successfully in testing. If exploratory testing is script-less and less documentation, automation requires all checking activities documented in detail. Even when we think of using notes taken during a test session to identify what automated and how it automated, it is not that simple. If you ask your tester to document all his activities with the level of detail that automation to use, it causes that exploratory testing in that session ineffective. If you only use exactly what noted by your tester, you will not have enough data.

Over time, we recognize that a combination between exploratory testing in a session and pair testing with an automation tester can overcome this obstacle. While one tester does exploration (investigation) on a product, he does whatever comes to his mind by thinking in reverse direction as how this piece of functionality would break, automation tester thinks of what can be automated to increase the effectiveness for next session.

Decide what to automate is the most important:

A good test case design for automation is that can run frequently, run fast, can use as many data as possible to perform the same action.
Below are some highlight criteria for your automated script selection consideration:

• Repetitive tests that run for multiple builds.
• Tests that tend to cause human error.
• Tests that require multiple data sets.
• Frequently used functionality that introduces high-risk conditions.
• Tests that are impossible to perform manually.
• Tests that run on several different hardware or software platforms and configurations.
• Tests that take a lot of effort and time when manual testing

Combine with the power of tool: two testers sit at the same machine to perform their tasks with their own objective is a solution (at least up to this point). The exploratory tester should be free and stays focus on his testing to gain the efficiency. Because of this reason, the automation guy should stay away asking many questions to avoid disturbs. So, sometimes he finds himself difficulty in understanding the context, or sometimes he needs to get back previous steps to document what he missed due to the exploratory testing happened too fast. Then, the support of the tool is necessary. You can use video tool to record all actions of the exploratory tester and playback when needed, for example you can use the step recorder of Windows 10 for that purpose. At MeU Solutions, a tool we have invented to support this kind of testing is One2Explore. A special feature in One2Explore called as “Exploratory In Pair” that allows you to invite another tester to perform the test session with you in parallel. When two testers join a session, one of them plays a primary role (who initiates the session) and another (who is invited) performs his tasks in limited permissions (it depends on the purpose at the point when setting up the invite. For example: if you choose “Invite to automate”, you are only able to select activities to automate, there is no opportunity to interrupt the session or interact to the application. One2Explore with the power of graphing the exploratory tests (*) allows you to select the activity quickly. All test activities conducted by the exploratory tester are visualized in a graph at the machine of the automation tester. This happens at runtime. At this point, the automation tester will choose what screen, flow and associated actions from the graph that will be used for automation. A set of test script steps are also generated and at this point, the automation guy also decides whether there are any keywords will be needed for his current test scripts. The keywords at this time are only prototypes that will be implemented later on. When the session ended, both of testers will sit to enhance the test scripts and make it reusable.

“Graphing the exploratory test” is another special feature to visualize the tests and explain them through pictures making it easy for everyone to understand which functions, integrations, flows, etc. have been tested. This feature is described in a paper which can be found at http://ebooks.iospress.nl/volumearticle/49972 or its demonstration video, you can see at https://meu-solutions.com/video-one2explore-graphing-feature-for-workflow-testing/ . Below is a portion of exploratory execution in the graph form.

 

Below is a workflow to illustrate how these two pairs work together with One2Explore (Of course, you can use another tool for One2Explore’s replacement)

We usually see failures in applying automation tests into exploratory testing because of differences in their philosophies. However, this novel approach is as a romantic marriage between the two. It has demonstrated the success of some projects being taken at MeU Solutions. The limit of the tool now is its ability in carrying out testing for mobile applications. Nothing is the best. “Learning challenges, perfecting solutions are things without being taxed or being taken away from ourselves.” – MeU Solutions

If you have any questions about One2Explore, please contact us at contact@meu-solutions.com