Think First, Then Test

(I have this statue on my desk)

You've just been given a project to test.  It's a big deal - important to the company, and important to you.  So you are ready to jump in and get going.  Should you just play with the first build?  Should you do some exploratory testing?  Should you check for that first bug you know is almost certainly waiting for you?  Should you start creating some automation?  Should you write a test plan first?

While people seem to naturally have a preference for action, I believe it's most important to think first, then test.

As Cem Kaner, James Bach and Bret Pettichord say in their excellent book Lessons Learned in Software Testing:

The difference between excellent testing and mediocre testing is how you think: your test design choices, your ability to interpret what you observe, and your ability to tell a compelling story about it.  The rest of testing is ordinary office work, for the most part.

They indicate four main categories of thinking that figure into the practice of testing:

Technical thinking.  The ability to model technology and understand causes and effects.  This includes things like knowledge of relevant technical facts and the ability to use tools and predict the behavior of systems.

Creative thinking.  The ability to generate ideas and see possibilities.  You will test only in ways that you can imagine testing.  You will look only for problems that you imagine can exist.

Critical thinking.  The ability to evaluate ideas and make inferences.  This includes the ability to detect and eliminate errors from our thinking, to relate product observations to quality criteria, and to build a compelling case for a particular belief or suggested course of action.

Practical thinking.  The ability to put ideas into practice.  This ability includes such skills as applying test tools and making test techniques and effort fit within the scope of the project.

Here are a few things to think about.

Before you test...

• Who are the relevant stakeholders for this project?
• What do they need from me most?
• How high should the "quality bar" be for this project?
• What are the risks?
• What could go wrong?
• What is the most important thing to test?
• In what order should things be tested?
• What tools do I need?
• What skills do I need to learn before testing this?
• Who could help?
• How much time do I have?

While you are testing...

• What have I learned during testing that I didn't know before?
• What seems riskier?
• What kind of overall quality am I seeing?
• What should I tell the stakeholders about it?
• Should I change my planned strategy?
• Do I need additional tools?
• Do I need more help?
• Do I need more time?
• What is most important for me to do right now?

After you test...

• What worked well?
• What didn't work as well?
• How could I have done that more efficiently?
• What didn't I catch, and why not?
• What should I do differently next time?
• What notes/documentation should I leave for next time?