^{Photo by Kenny Louie}

“Children are born naturalists. They explore the world with all of their senses, experiment in the environment, and communicate their discoveries to those around them.” ^[1]

What is ET?

Exploratory Testing is the nature of testing^[2]. The pure beauty of ET is that it starts without specific test plan and tester comes up with ideas or assumptions during testing execution. Then test design and test execution happen at the same time. It’s the true symphony of efficiency!

What is my understanding of ET?

For me, ET is an elegant testing approach with a free spirit, to think about different possibilities, to discover the unknown, to collect effective information and to share new knowledge efficiently.

What is my practice of ET?

Since the beginning of last year when I failed to use Calabash for mobile testing, I mainly focus on improving my ET skill. I want to establish a practice for the purpose: To have a concrete structure of ET and to adapt change easily based on it.

Combining the purpose and my understanding, I tried some practices and made some changes. It becomes the current one I’m satisfied with. First of all, please remember it’s definitely not the “best practice”. I still keep striving for a better practice, your feedback could help me as well 😉

Back to my practice of ET, each time I proceed with 3 important stages (a.k.a. 3G stages): Get context, Go explore and Give feedback.

Stage 1: Get context

Time estimation: 15 min

On this stage, the key is to think different. By asking PO, stakeholder, designer, developer, etc… who already have the information of context. For instance, if the testing target is a new feature:

• To understand the problem behind: What’s the problem to solve?
• To understand the request of developing this feature: What’s the acceptable result look like?
• To understand the detail of implementation: Which kind of user cases are considered and checked already?

With this information in mind, I will think about that the implementation does not only meet the request but also provide the functionality to solve the problem:

• Implementation versus Request: What are the misunderstandings and ambiguities?
• Implementation versus Problem: What are the blind spots and missing checks, could potentially cause the risk of malfunction?

Normally I prefer to take some factors and put them in the questions, in order to generate some testing ideas. Some factors could be considered:

• Environment: region, timezone, language…
• Hardware: device, platform, OS, screen size…
• Aspect: usability, security, performance…
• …

For example:

Region x Implementation versus Request: Does this feature need to be available for all customers or only for a specific region?
OS x Implementation versus Problem: Does this feature work on iOS 7 devices?

Testing ideas:
[ ] Support all regions?
[ ] iOS 7

Stage 2: Go explore

Timebox: 30 min

This stage is the most exciting one. The key is to discover and to collect. Before starting exploring, a time box is rather important, which forces me to proceed with the next steps. Otherwise, I could probably explore forever far around and way deeper.

Remember the to-do list earlier I got in the first stage? Now it’s time to make hands dirty using the ideas, I call them seeds, as initial starting points to discover the unknown. Following and focusing on one seed to perform some checks, I will start to collect new seeds:

• Relate to current seed, add it as a child seed
• Not relate to current seed, add it as a new seed

Once I got a finding or a question, I will make a quick note with the necessary screenshot and video record.

After one seed is finished, I will continue to explore anther seed. One seed at a time, seed by seed until timeout. You probably have a question in your mind “When one seed is done, what to pick as the next seed?” I have a rule of thumb for priority: seed on top of the list has higher priority to explore.

After 30 min, my to-do list looks like:

Testing ideas:
[?] Support all regions?
    [x] Europe
    [?] Asia: no response from API
        => Question 1
[-] iOS 7
    => Issue 1
[-] No interent
    [x] Error message
    [x] retry when internet is back
    [x] app restart when no internet
[?] Animation
    => Question 2
[ ] Performace of new API request?

Question 1: need API response for Asia?
Need check with backend team: this feature is not available for Asia users?
<Screenshot of API request and response>

Question 2: Refresh animation feels laggy
Need check with designer and developer: make a nicer animation

Issue 1: crash on iOS 7
<device info>
- <step 1>
- <step 2>
- <step 3>
<crash stack traces>

Stage 3: Give feedback

Time estimation: 15 min

The key to this stage is to share knowledge. I’ve some questions and issues on my list collected from the previous stage and I know who I need to contact. Can I just throw my feedback on their face directly? I hope I can but apparently nope.

Regarding the questions, it needs to have a proper conversation to remove misunderstandings and ambiguities:

• What’s my opinion and suggestion?
• What’s your opinion and suggestion?

Regarding the issues, it needs to have a proper discussion to evaluate the risk:

• What can be fixed?
• What is out of scope?
• What needs to be improved in the future?

What else?

Basically, it’s the core part of my lovely ET practice:

• 3G stages: Get context, Go explore and Give feedback
• 4 actions: Think, Discover, Collect and Share

Although there are still a lot of topics I’d like to expand, I don’t want to make one post too long too boring. I will probably write a series of ET topics in the future, like:

• How to think different and some factors to consider?
• How to collect and share findings efficiently?
• How to shorten feedback loop in development?
• How to train myself be always ready to be interrupted?
• How to pair with developers?
• …

Testing is like holding a torch to light up the road in darkness. You could clearly see the way to go, make a map, and mark thorns and flowers.

[1] The Audubon Nature Preschool
[2] Cem Kaner: A Tutorial in Exploratory Testing