April 2, 2006

What is the Correct Ratio of Development Time to Test Time?

Recently, I read this question:
"I just wanted to get a feel for how much time do we spend testing vs developing?  If the development staff vs testing staff ratio is 3or4:1, then do we assume development time vs test time is 3or4:1?"
There's no inherent preset relationship between development time and testing time.  And there's no inherent preset relationship between development/test staff ratio, and development/test time ratio.

CASE 1
Suppose you had this new requirement:
Your product, which was never previously tested on Windows XP SP2, now must support that OS version.
Development takes 1 day (Windows XP SP1 worked fine, they had to change some text in the "System Requirements" doc, but don't believe they have to do anything else)

Can you predict how long testing should take?
  • For some companies, they may take 1 day of casual testing.
  • For other companies, they may take 6 months (perhaps they are in a regulated industry, and the cost of missing a bug is very high)

CASE 2
Suppose the company wanted to change their About box.
Replace the former text-only dialog, with a beautiful graphic.
Since the developers aren't very good graphic artists, it takes them 3 days to learn the new graphics tool, 3 days of trial-and-error with the Marketing Department to make it look "just right", and 2 days to get it integrated into the build.

Can you predict how long it might take to test?
  • In some shops, a Junior QAer might spend an hour or less checking it out.
  • In other shops, since the new company logo is embedded in the graphic, and it must precisely match the new size and color specifications now required by Marketing, it might take longer.
Clearly, the time required to test something is contextual - it depends on factors that may have little or nothing to do with how long it took to develop that feature.

Also consider:
  • what counts as Development?
  • what counts as Testing?
  • If we were going to perform regression testing anyway, does that count as "zero" additional testing hours?
See also:
http://blogs.msdn.com/alanpa/archive/2008/05/24/test-to-developer-ratios-you-re-probably-doing-it-wrong.aspx