Scripting Development Mindset in Automation Testing
Here is the next article in a series of practical mindsets for automation tester. In previous, we showed 10 Good Practices to be an Expert in Automation Testing and A Scripting Framework for Automation for automation tester.
Writing automated tests is rather easy but writing good and maintainable tests are much tougher. This is especially true when your test projects grow in size and complexity. In this article, we will focus on mindsets of automation tester in scripting development which can help tester structure tests in projects with success. Which can help to reduce test duplication and complexity while making tests easier to organize.
1. If test data is created, delete them before the test script finish.
In perfect condition, we expect to create an independent test script which can run and give output without depending on any other script. It means its steps and data need to be isolated from other test cases. By this way, we can run these test cases whenever we need and wherever we want.
In order to complete this, we need to let test script cover pre-configure data itself and make sure they will clean data completely when test case has done.
Above steps is a popular structure of a test script. It split into 3 parts:
Part 1: Prepare data for test script running
Part 2: Run test scenario and verify checkpoint
Part 3: Clean data which are created in part 1.
There is nothing to discuss if the test case is run successfully at all steps. But what happens if a step was failed. Will the next steps need to run or skip?
When a step is failed its mean test script is failed. So, continue running the rest steps just make waste time, but skip the rest steps will make test script leave the trash data which can make an impact to next scripts. So, what are our solutions to make our script both save time in a failed case and clean data completely?
After a long time working in automation testing, we want to propose to you an efficient flow to run the script like this :
|
With the above flow, when test script is failed at any step, the rest steps will be skipped for saving time but cleanup data steps always run to make sure return clean environment for next test script.
2. An automated script will be run without human intervention
When we develop a test script we must always aware that this test script will run automatically without any human intervention. So, we must manage all unexpected condition in our code.
Example:
- The slow network can make step failed. So, we need to wait for the object for fully loaded in a period of time (usually 30-60 seconds)
- Suddenly popup from another app makes our AUT lost focus => We need to define a recovery scenario to auto-detect the strange popup to close it and reactive AUT window to run the test.
- AUT crash => define recovery script to reopen AUT
- Longtime running can lead to memory leak => restart AUT after a set of the test script to refresh running environment.
3. The business function can be changed anytime.
Nowadays, most of the project is developing in an Agile process which is always ready to change requirement, so selecting a framework to adapt to change is very important. By our experience, using a hybrid framework which combines keyword driven and data-driven is the best match for this case. When requirement change we just need to update keyword, no need to change anything from test cases. It will help to save a lot of time for maintenance test script when AUT change.
4. Software development is different with automation script development
Both software development and automation testing are using programing language to develop but they have some different requirements.
| # | Software development | Automation testing |
| 1 | We must optimize our code as much as possible. | A test step can take few seconds or minutes to complete so optimize code save milliseconds won’t have much meaning. |
| 2 | Developers have high skills in development, so their code may use a lot of complex syntaxes, complex techniques like Recursive, LINQ, lambda, annotation …. | Automation tester may have fewer skills than the developer, their script may be read by whom have a limited programming skill, so they must use a simple syntax, details comment. Don’t use complex syntax/ technique |
| 3 | Some developer doesn’t want to push comment in their code. They naming variable and function name which have enough meaning to understand their code. | Must need the comment to explain test steps |
5. Always ask yourself how much time for a step
We have a thousand of test script need to be run for each iteration, so saving more time for each step is better. So, we must optimize time for running the test script by:
- Use dynamic waiting time instead of hardcode sleeping in specific time.
- Don’t waste too much time to check unnecessary/duplicated checkpoint.
- …
6. A next step can be a checkpoint for the previous step.
I assume that we have a test case as below:
| # | Step | Checkpoint |
| 1 | Create user has name User A | Verify the user is created successfully |
| 2 | Rename the name to User B | Make sure renaming successfully |
| 3 | Delete User B | Verify delete user successfully |
Normally, we develop the script by call keyword like this.
If I rewrite test script like this
This rewrite still ensures sure the correctness of test case because if we can rename User A to User B, it means user A was created successfully. Or if we can delete user B, it means that User A has been renamed to User B successfully. By this example, we can see step ‘Rename’ became a checkpoint to step ‘CreateUser’ or step ‘Delete’ became a checkpoint of step ‘Rename’. Automation tester must understand this technique to reduce test steps in a test case to optimize run time of test script.
Wrapping Up
Hopefully, after reading this you have a better idea, if you didn’t before, about the mindset around automated testing and the goals we are trying to achieve when testing an application.
