How long will it take to test? Sometimes when I hear that question, I cringe.
Don't bother me, I'm Testimating!
(not actually me)
Estimation is the calculated approximation of a result which is usable even if input data may be incomplete or uncertain. [Wikipedia]
Guesstimation is an informal English word derived from guess and estimate, first used by American statisticians in 1934 or 1935. It is defined as an estimate made without using adequate or complete information, or, more strongly, as an estimate arrived at by guesswork or conjecture. [Wikipedia]
Testimation is what you are forced to do when someone asks you to quickly predict how long it will take to test something - something that you've never done before, or seen, or even have any idea about what might actually be involved. [Me] copyrightjoestrazzere
Does this scenario sound familiar?
Product Manager: Hi, Joe. We're putting together estimates for building a Zerble feature for the Framis product. How long do you think it would take to test?
Joe: Uhm, what's a Zerble?
Product Manager: Well it's sort of like a Whatsit, only different. Frankie, the new guy in Development, thinks it will only take him 3 months to finish.
Joe: Are there any written Requirements I could read?
Product Manager: No.
Joe: Has Frankie designed it yet?
Product Manager: No.
Joe: When do you need an estimate?
Product Manager: Now.
Joe: Hmm...
Ok, so I'm exaggerating a bit.
Usually, you get some vague idea about what might be required, with a promise by the Product Manager to be more specific somewhere down the road - you know, after they actually talk with the customer and find out what they need. And if you are lucky, the Developer may have some concept of the design he will choose - likely something between a quick hack and a rewrite of the entire system - he'll tell you more right after he's finished deciding which technology to use, and after he's finished coding the thing. And you usually don't have to provide your estimate immediately, end-of-day is usually good enough, and it's only 4:30 now.
What you'd like to do is estimate something like "Somewhere between 1 week and forever." But, you are a serious professional, and folks expect a thoughtful, scientifically-derived answer from you. So instead, you answer "If I have to guess right now, I'd say 2 months, but I can't have any confidence in that until I learn more. I'll give you a more accurate estimate when I understand the Requirements, and have some idea of the Design."
And of course two things inevitably happen:
- A formal project plan is created showing 3 months of development effort followed by 1 month of testing.
- Someone higher up the food chain instructs the project team to "get creative" about doing better than 4 months total. Because, after all, they promised the customer that it would be delivered by the end of this month.
I've been trying to think of a good analogy I could use to better explain to folks how inaccurate some of these Testimations must be, and what it feels like to be part of one.
Imagine you are a professional proofreader. Your boss asks you for an estimate.
Boss: I'd like you to proofread a book that Ted is writing. How long do you think it will take?
The last book you proofread was a book of short stories for children. It took you 4 months to proofread, so you might say "Four months".
Boss: Great! I'll tell the printers that we'll have the book to them in five months - that should give us plenty of time.
You have a lot of other work to do over the next few months, and you won't be able to enlist any help. So you mentally go through ways you can shuffle things around, and you agree.
Boss (one day later): Oh, I forgot to mention, Ted says he's a bit delayed, but he promises to get the first draft of the book to you in five or six more weeks. Since you estimated four months, and you are a really good proofreader, that won't be a problem, right?
Boss (two days later): Did I mention that the book is about the history of abstract painting in the 20th century? Yeah, I know you don't know anything about painting, but it's still a book, right?
Boss (seven weeks later): I spoke with Theodopolous, and he's almost finished. Should be just a few more days after he returns from his vacation in Greece next week. By the way, he decided it would be a lot quicker if he just wrote it in Greek. That's not a problem is it?
Boss (four weeks later): Theodopolous said he's running a bit behind, so he decided to get the book to you in two parts. He'll have his part ready in no more than one month. His partner, Frank promised to have his part done at the same time.
Boss (three weeks later): Good news! Theodopolous has his part - Chapter 1 - done ahead of time! He'll send it to you tomorrow. By the way, Francois has decided it will be quicker for him if he just writes Chapters 2 through 36 in French, since he doesn't write Greek or English. Oh, and it's going to take him a bit longer than expected. He's estimating four more weeks.
Boss (one month later): The printers said that they absolutely need that book in their shop by the end of this week, since the presses will be busy printing census forms for the following six months. Ted says that shouldn't be a problem - he's already met his committment. Let's be creative and get it to them in time, ok?
Boss (later that day): I'd like you to proofread a book that Alice is writing. How long do you think it will take? I promised to have it to the printers before they get busy with those census forms next week.It's pretty clear that the accuracy of a test estimate depends on many factors - some of them out of our control. With more information, we can be more accurate. With more history and practice performing the same task, we can be more accurate. With more control of the situation, we can be more accurate. With more flexibility, we can be more accurate.
But with little information, no history, no control, and no flexibility, our test estimate is nothing more than a guesstimate.
I can always give you my Testimation Guesstimation right now. But if you want a real Testimation Estimation, you might have to wait a little while.
This article originally appeared in my blog: All Things Quality
 | My name is Joe Strazzere and I'm currently a Director of Quality Assurance. I like to lead, to test, and occasionally to write about leading and testing. Find me at http://strazzere.blogspot.com/. |
