Bruce Asks All Things

Ask All Things

Bruce asked a great question:

We have a large number of tests that we would like to automate.  However, it takes time to automate each test, and we have several thousand tests that could be automated. 

Is there an average or maximum number of tests that QA people automate using tools like Selenium or Ranorex?  Thanks for your help.

Sincerely, 

Bruce 

Nice question, Bruce! And one that many folks contemplate when they encounter a large system that needs a lot of test automation.

At one place I worked, we automated over 8,000 tests.

In that particular shop the UI and much of the underlying code was very, very stable. New functionality was added, but it was virtually always added in new instances, and seldom involved changes to existing instances. Thus automated regression tests were able to pay back the investment, and maintenance wasn't overly burdensome.

I know of no industry averages or maximums. Even if there were such averages, I don't see how it would be of much help for your individual case.

Instead, you should consider the full cost of automation in your shop (including initial setup and ongoing maintenance) against the benefits you expect to receive.

Some factors to consider: copyrightjoestrazzere

• Do you have sufficient expertise already on staff to create this automation, or will you need to go outside your organization to get it?
• Do you have time in your schedule to plan, design, develop, test (yes - your automation needs some testing!), document, and deploy your automation?
• How much ongoing maintenance will this automation need?
• How much is the current lack of automation costing your company - both in terms of bugs escaping to Production, and the cost of manual testing you must perform due to the lack of automation?

Many companies choose to start small. Automate only the parts that matter most (perhaps the riskiest or most critical parts of your system), weigh the costs versus benefits, and then decide what to automate next (if anything).

Some companies choose to plan automation for new systems, but only tactically go back and automate legacy systems as time permits.

Other companies choose to outsource the initial automation effort, then bring the maintenance portion in-house.

Let's throw this question to the readers - Do you know of an average or maximum number of tests? Do you know of other factors to consider as you think about automating a large system? Have you faced a large automation effort like this? If so, how did you handle it?

-joe




Do you have questions? Use the new "ASK ALL THINGS" widget over on the right-hand panel. Send me questions about anything:

• about the testing profession
• about test automation
• about bug tracking
• about being a Manager
• about testing and QA jobs
• about quality
• anything!

I'll read through the questions, pick some that not only interest you and me, but questions that I think will interest others. Together we can not only get you the answers you need, but we can provide others with some useful information as well.

Ask All Things!

---

This article originally appeared in my blog: All Things Quality

	My name is Joe Strazzere and I'm an experienced Quality Assurance professional. I like to lead, to test, and occasionally to write about leading and testing. Find me at http://AllThingsQuality.com/.