How to improve eCommerce website testing with exploratory testing

In the previous article, we talked about using Exploratory testing in game testing effectively. Today, we will learn and practice more about how to improve e-commerce app testing with the Exploratory testing – this practice will base on the [Mindmap] Testing ideas for E-commerce Website Testing that were posted in the previous article as well.

E-commerce websites nowadays are becoming popular and popular in our daily life, it’s playing an essential key in the economics, finance – banking, marketing, sales, … it has many advantages compared to the traditional shopping way, such as time-saving, easy access to products worldwide, quickly compare the prices, buy the product whenever you want. So, a “good” E-commerce website is a key to its success, and it’s vital for QA to test that the application smoothly under various conditions and make sure that it doesn’t have significant defects that can harm the business.

Why E-commerce Testing is Imperative

• Security breaches: The quality of e-commerce platforms have increased steadily years over years, but there are still many risks in the application. Nowadays the system not just only display the number of items, payment gateways, and shopping carts, rather they encompass an extensive set of features including reporting & analytics, catalog management, marketing modules, campaign management, customer profiles management, etc. that making the application more and more complicated. More complex systems are harder to test and therefore are more likely to have untested portions, and complex systems have more lines of code, and thus security bugs may exist.  Moreover, the e-commerce becomes more popular, and it’ll be a target of the hacker.
• Poor performance: While the popularity of eCommerce platforms continue, the past few years also have been witness to many high-profile website glitches and crashes, especially during the high-traffic holiday season. From Walmart, Belk, Tesco to the mighty Amazon, all have experienced significant glitches at crucial times, resulting in massive losses, disgruntled customers, and some severe media bashing. So, the load testing/ performance testing is vital for eCommerce websites.
• Poor compatibility: The market is flooded with a lot of mobile devices that customers use to access the website, so, the sites may be displayed differently in different browser versions in different OS/devices, which often leads to unexpected functional defects. E-commerce app needs to be compatible across a range of browsers and devices.
• Poor User Experience: The primary distinction between dev and tester is state of mind. While developers write pieces of code that later become features in the application, QA is expected to understand how real users think, engage customers to learn business scenarios, and confirm that the application satisfies customer needs.

How to improve e-commerce website testing with exploratory testing

To create high-performance, reliable e-commerce websites, we need to test it carefully and rigorously. E-commerce websites find a lot of high risks so we can not rigidly apply the usual methods to perform tests that need to be flexible.

We need a method for getting rid out of framework thinking, liberating the mind to describe the various aspects of testing, finding out hidden issues, and issues related to the software’s platform. So that we may find many bugs in the system as early as possible, and additionally considering the agile market, including faster release cycles, frequent product updates, price changes … Exploratory testing can meet the above requirements.

As the name implies, Exploratory testing is a test approach that allows you to apply the skills, and techniques of a tester (QA) most effectively. During the QA test, learn about the behavior of software applications, start creating test plans or test scenarios, and make decisions about what to test and analyze. Eliminate the time to check it is reasonable. In other words, Exploratory testing “The explorer can do any combination of learning, designing executing and interpreting at any time”

How do we do the testing strategy with exploratory testing

Testing E-commerce Websites requires knowledge of web testing techniques and the e-commerce domain. So, we list all the necessary ideas at below as references.

Testing techniques

• Functionality + Exploratory Testing
• Regression testing
• Usability Testing
• Security Testing
• Performance Testing
• Compatibility Testing
• Integration Testing– All the third-party integrations (like payment processors)
• A/B testing
• Automation Testing

The e-commerce domain

Most E-commerce Websites share a general common theme and structure as below, of course, there are many other pages on a typical e-commerce website but not listed here.

• Front-end pages: Homepage, Search Results Page, Product Details Page, Shopping Cart page, Product Category page, Checkout and Payment Gateways page, Login Form Page and Accounts Pages, Change Languages, etc
• Back-end pages: Product Search Service, Content Service, Booking Engine, Payment Services, Accounts Services, etc

At MeU – We translated all above requirements into practical testing checklist points (the checklist includes examples of some of the most common e-commerce features that would likely be flagged for testing) and yet we created an e-commerce website checklist by mindmap to help us organize the test ideas and execute exploratory effectively. You can refer to it as your testing reference.

How we apply the checklist in a real e-commerce website

At MeU – we will test and practice with the e-commerce website — Shopee. Shopee as a marketplace helps sellers and buyers find each other, deals, pays for products. It is a leading online shopping platform in Southeast Asia and Taiwan (Singapore, Malaysia, Indonesia, Thailand, Vietnam, Philippines and Taiwan).

Analyze ‘Shopee’

• First of all, we look at the Shoppe GUI and features which can help us to have the overview of the whole system, mapping the system features and its structure to our current checklist and make the first update to the checklist to make sure that the new checklist is reflecting the Shopee functions and features.
• Secondly, we will analyze the Shopee website deeper by doing and performing each function to cap main features to focus. Main functions that we identified for the buyer are product details, product search, product purchase, product payment, shopping cart, track order. For the seller, we will test how to upload products on the website, promote products to customers, sale and how to get income … Besides the functions that are indispensable, the user interface is also important in attracting customers so we also test it. On the other hand, we need to check some other things like SEO – Search engine optimization- It’s easy to find the website or not when users search for relevant keywords on search engines. Information security – User information is secured, no data is lost, transaction information is safe; website performance – determine the level of response, bandwidth, reliability, or scalability of the system under a certain amount of work/access.
• Thirdly, brainstormed a first draft top-level list of the risk that e-commerce website may encounter from our historical database, from Google, … apply our quality evaluation models analysis.
• Fourthly, at MeU – we will use the Xmind tool to generate and manage test ideas and use One2Explore to perform exploratory testing. With One2Explore capabilities, the exploratory testing was more effective (kept track of all application behaviors that associate with its status and relevant data). It helped us to structure, plan, track, report and debrief the session in a picture to ensure the goal covered. One2Explore was integrated with One2Test (Testing Dashboard) and JIRA to convey the quality messages to key stakeholders (The quality info such as bugs per features, bug per charters, numbers of charters (executed, planned, blocked), bugs trend chart, … were visualized into a dashboard which helped us to make decisions on product quality. As for test performance, we use JMETER because JMETER is open source, free to the user and simple interface. For SEO, we use the Seoquake tool. It is a handy tool in the browser that helps us – who will test the website know that the site has SEO or not, the number of visitors, the site is easy to connect with users or not,… For automation testing, we use One2Automate, it is an open test automation framework which was built by testing professionals.

Note: We will not test the payment gateway. For the testing environment, we focus on popular browsers such as google chrome, firefox, internet explorer (IE), CocCoc browser on the mobile as well as on the laptop.

So, let’s see with examples how to test and e-Commerce Site as below, in the mindmap format:

And our Test Result

ID	Bug’s Title	Description	Priority	Environment
1	Time in the flash sale is unsynchronized	The flash sale time on the main page is unsynchronized with the flash sale page as well as the time when scrolling the flash page down	Low	Windows 7, 10     Browsers: Chrome, CocCoc, Firefox and Internet Explorer (IE)
2	The cart icon doesn’t display the correct number of products when users add or delete products	The number of products on the cart icon doesn’t update until users reload the page. It is the same problem with deleting a product.	Medium
3	Only one product can be deleted on the shopping cart icon during a session	when users hover to the cart icon to delete a product, users can’t delete more products on the cart icon as well as in the cart. It is only deleted when users reload the page.	Medium
4	The language is unsynchronized between the main page and the order page	It happens when users register an account with an email	Medium
5	The error message does not indicate the correct error when having a duplicate username or shop name	It only shows the message” Server error”	Medium
6	Buyers page should be able to change the English language like the seller page		Medium
7	More ………………[hided]