The Internet of (Insecure) Things

I know what you did, and where you are

IoT - the Internet of Things! 

Everything is connected to the internet! Refrigerators talking to toasters! Washing machines talking to garage door openers!

And were these Things tested well? (Probably not) Can they be hacked? (Almost certainly)

This one in particular caught my eye - GPS watches. @copyrightjoestrazzere

How GPS Watches Can Put Kids In Harm's Way

https://www.cbsnews.com/news/gps-watches-for-kids-are-easily-hacked/

Personally, I don't understand the appeal of the devices in the first place. I get that some folks are helicopter parents, but I don't understand the need to know exactly where your children are every second of the day.

Putting that aside, you must assume that if you can track a device, others can as well. And a device advertised for tracking children is simply a disaster waiting to happen, in my opinion. 

Be careful here, folks. I'm not suggesting you should walk around with a tinfoil hat. The Internet of Things has a ton of potential. But every day we hear about more and more devices with stupid default passwords, or that are trivial to hack. Think it through - if your device were hacked, what could the worst outcome be? Sometimes it's a lot worse than having the freezer tell the clock radio about that cheesecake you ate whole last night.

Stay safe out there.

---

This article originally appeared in my blog: All Things Quality

	My name is Joe Strazzere and I'm an experienced Quality Assurance professional. I like to lead, to test, and occasionally to write about leading and testing. Find me at http://AllThingsQuality.com/.

Patriots Training Camp 2017

Patriots Training Camp 2017

This morning, I took in the New England Patriots Training Camp. It was a hot and humid morning, but fortunately I was able to find a bit of shade.

I hadn't been since 2011. Some things have changed, but a lot of it is the same. A few observations:

• Tom Brady is still there, and he still looks great. (On Thursday, he'll be 40! I'm getting old.)
• Lots more activities for kids and adults. 
• Tons of workers there, working security and the activities, and selling water, soda, Gatorade, and frozen lemonade. One worker that I remember from way back is a guy who sells sodas and uses players' names as part of his pitch ("It will stop your thirst like Butler stops touchdowns!"). Funny stuff.
• The defense looked really good today, particularly the DBs. Lots of good coverage and some interceptions.
• The offense didn't look as good. Lots of mistakes.
• Julian Edelman and Stephon Gilmore got in a bit of a fight about 3/4 of the way through the practice. They were both tossed. Belichick yelled something like "Just get out!"
• Rookie Deatrich Wise knocked down a pass at the line - he looked pretty good
• Brandin Cooks looked pretty fast
• Neither Jimmy Garoppolo nor Jacoby Brissett looked great today.
• At one point, Belichick threw a blue blocking pad at Brady while he was scrambling around. Brady completed the pass anyway. I'm pretty sure that sort of thing won't actually happen during a real game.

A fun time all around.

Go Pats!

---

This article originally appeared in my blog: All Things Quality

	My name is Joe Strazzere and I'm an experienced Quality Assurance professional. I like to lead, to test, and occasionally to write about leading and testing. Find me at http://AllThingsQuality.com/.

Other Places You Can Find Me

I also participate and answer questions at: 

The Workplace StackExchange

https://workplace.stackexchange.com/

and

Software Quality Assurance & Testing StackExchange

https://sqa.stackexchange.com/

---

This article originally appeared in my blog: All Things Quality

	My name is Joe Strazzere and I'm an experienced Quality Assurance professional. I like to lead, to test, and occasionally to write about leading and testing. Find me at http://AllThingsQuality.com/.

A Free Online Test Conference

I've signed up to attend this free online test conference.

Some terrific speakers including Joel Montvelisky, Alan Page, Matt Heusser and others. You've seen some of them in my People in Testing and What People Are Writing tabs.

Some great topics like Keeping QA Relevant, Mobile Test Labs, Crowd Testing and Automation.

And of course it's free!

How about you?

http://www.onlinetestconf.com/

---

This article originally appeared in my blog: All Things Quality

	My name is Joe Strazzere and I'm an experienced Quality Assurance professional. I like to lead, to test, and occasionally to write about leading and testing. Find me at http://AllThingsQuality.com/.

Perhaps They Should Have Tested More - The Schiaparelli Mars Lander

Schiaparelli Lander

Bugs have consequences! 

The wrong bugs can crash your Mars lander, cost you a lot of money and cost you a lot of confidence in your planned Mars missions, as the European Space Agency found out.

Back in October 2016, the ESA attempted to land the Schiaparelli lander on the surface of Mars. But software bugs caused the craft to crash and to produce a rather impressive crater.

Here lies Schiaparelli

Recently, an independent investigation has concluded that insufficient parachute modeling, inadequate handling of alerts, and an “insufficient approach to Failure Detection, Isolation and Recovery and design robustness" were to blame for the failure. #copyrightjoestrazzere

Essentially, the software thought the spacecraft was closer to the ground than it actually was, and released the parachute and shell early. When the control system shut down, the craft was still 3.7 km in the air - oops! This resulted in a 34 second free-fall, subsequent crash at a speed of 540 km/h, and the destruction of Schiaparelli.

In spite of the failure in the planned soft landing, the Europoean Space Agency declared the mission "a success". Maybe they were just really excited about the brand new Martian crater they discovered?

Perhaps the European Space Agency should have tested more.

See Also:

http://www.space.com/34821-exomars-lander-crash-measurement-error.html

https://www.theverge.com/2017/5/27/15704644/esa-mars-schiaparelli-lander-crashed-software-glitch

https://en.wikipedia.org/wiki/Schiaparelli_EDM_lander

http://www.sciencemag.org/news/2016/10/mars-lander-crash-complicates-follow-rover-2020

https://www.nytimes.com/2016/10/21/science/space/no-signal-from-mars-lander-but-european-officials-declare-mission-a-success.html

http://www.esa.int/Our_Activities/Space_Science/ExoMars/Schiaparelli_landing_investigation_completed

http://exploration.esa.int/mars/59176-exomars-2016-schiaparelli-anomaly-inquiry

---

This article originally appeared in my blog: All Things Quality

	My name is Joe Strazzere and I'm an experienced Quality Assurance professional. I like to lead, to test, and occasionally to write about leading and testing. Find me at http://AllThingsQuality.com/.

Songs about Testing, QA

I was looking for some music related to Testing or QA to use in a presentation.  The pickings are slim, but I found these.


Works on My Box
- Art Leonard
http://artleonard.com
http://artleonard.com/audio/ART_LEONARD-Works_on_My_Box.mp3

Art Leonard is a Seattle-based Christian, radio and novelty songwriter / performer. His software engineering anthem, "Works on My Box", made a big splash at a major software development company and is now played to new employees at orientation.

Feature Creep

- Geordie Keitt

http://tester.geordiekeitt.com
http://tester.geordiekeitt.com/2009/08/cast2009-feature-creep-song-video/

I performed this a month or so ago at CAST2009 in Colorado Springs, where Becky Fiedler recorded it. The intro to this song went, “This song is written from the perspective of a piece of bloatware that used to be sleek and clean.” I did this during a Lightning Talks session, meaning I had to bring it in under 4 minutes. That’s why there’s little time for dramatic pauses…

Black Box

- Geordie Keitt

http://tester.geordiekeitt.com

http://tester.geordiekeitt.com/wp-content/uploads/2008/03/black-box.mp3

Cem Kaner and James Bach presented a course on Black Box Software Testing that I took circa 2003. I performed this song, Black Box, to wrap up the training.

Rapid Tester

- Geordie Keitt

http://tester.geordiekeitt.com

http://tester.geordiekeitt.com/wp-content/uploads/2008/01/rapid_tester.mp3

I this wrote on the occasion of attending my first RST class at James Bach’s Satisfice world headquarters in Front Royal, VA. I’ve edited it slightly since then as my understanding of the material has grown, but the essence remains the same. It borrows pretty much everything from Steve Earle’s beautiful song “Someday”.

Not on The Test

- Tom Chapin and John Forster
https://www.youtube.com/watch?v=8dAujuqCo7s

Ok, so this one isn't really about QA or software testing.  It still made me laugh and has a few poignant lessons about metrics and relying solely on a script, rather than on rational thinking:  ".. Don't think about thinking, it's not on the test!"

Bugs
- Ron Brown
http://www.songsforteaching.com/intellitunes/bugs.htm
http://www.songsforteaching.com/intellitunes/clips/bbbbugs.mp3

Another one that isn't really about software.  But it's a cute song, and how can you not like a song that starts off "B-b-b-bugs, bugs, bugs"?

Piece of Crap
- Neil Young
https://www.youtube.com/watch?v=Ovum-GjYWKQ
http://web.archive.org/web/20040131135434/http://www.se.fit.edu/people/James/misc.html

While it's not really about software, this song fits well. I was particularly looking for a music video that James Whittaker and some of his students performed a while back. I tried all the searches I could think of, but no luck. Fortunately, my friend Martin sent me the above web archive link.

Know of any others I could add to this list?

---

This article originally appeared in my blog: All Things Quality

	My name is Joe Strazzere and I'm currently a Director of Quality Assurance. I like to lead, to test, and occasionally to write about leading and testing. Find me at http://AllThingsQuality.com.

Double the Bug Bounty, Double the Fun - Microsoft

It's fun to find Office bugs

Like to hunt bugs? Want to have some fun, and perhaps make some money at the same time?

From March 1, 2017 to May 1, 2017, any eligible vulnerability submitted for Microsoft Office 365 Portal and Microsoft Exchange Online will be eligible for double rewards. 

Hence, any qualified vulnerability found in the domains below will receive up to $30,000 if it’s submitted between March 1 and May 1, 2017.

The specific domains include:

• portal.office.com
• outlook.office365.com
• outlook.office.com
• *.outlook.com
• outlook.com

For more information see: https://technet.microsoft.com/en-us/dn800983.aspx

---

 This article originally appeared in my blog: All Things Quality

	My name is Joe Strazzere and I'm an experienced Quality Assurance professional. I like to lead, to test, and occasionally to write about leading and testing. Find me at http://AllThingsQuality.com/.

All Things

---

This article originally appeared in my blog: All Things Quality

	My name is Joe Strazzere and I'm an experienced Quality Assurance professional. I like to lead, to test, and occasionally to write about leading and testing. Find me at http://AllThingsQuality.com/.

Minimum Acceptable Quality Levels

I like this:

Tell me what your minimum acceptable quality level is and I'll tell you when you can have it.

-Dilbert

http://dilbert.com/strip/2016-02-21

---

This article originally appeared in my blog: All Things Quality

	My name is Joe Strazzere and I'm an experienced Quality Assurance professional. I like to lead, to test, and occasionally to write about leading and testing. Find me at http://AllThingsQuality.com/.

State Of Testing Survey - Last Chance

It's your last chance to participate in the annual "State of Testing" survey. Won't you help?

Our friends at PractiTest and Tea Time with Testers are running their annual "State of Testing" survey again.

Last year, almost 900 testers responded, and helped to paint a clear picture of our profession. We learned about salaries, the size of test teams, the experience of testers, what testers actually do (besides testing of course), and lots more.

This year, almost 1000 testers have helped so far. How about you?

Please go to http://qablog.practitest.com/state-of-testing/ and fill out the survey. It's fast, it's easy, and it will help us all!

---

This article originally appeared in my blog: All Things Quality

	My name is Joe Strazzere and I'm an experienced Quality Assurance professional. I like to lead, to test, and occasionally to write about leading and testing. Find me at http://AllThingsQuality.com/.

A Dilbert Bug Report

http://dilbert.com/strip/2016-01-06

Pretty funny, but not a very effective bug report!

• Where are the steps to reproduce the problem?
• Where are the expected results?
• Where are the actual results?
• And of course, what is the Priority and Severity?

---

This article originally appeared in my blog: All Things Quality

	My name is Joe Strazzere and I'm an experienced Quality Assurance professional. I like to lead, to test, and occasionally to write about leading and testing. Find me at http://AllThingsQuality.com/.

What's The State Of Testing In 2016?

It's that time of the year again.

Our friends at PractiTest and Tea Time with Testers are running their annual "State of Testing" survey again starting Thursday.

Last year, almost 900 testers responded, and helped to paint a clear picture of our profession. We learned about salaries, the size of test teams, the experience of testers, what testers actually do (besides testing of course), and lots more.

I strongly urge you to go over to http://qablog.practitest.com/state-of-testing/ and help. I know that I will!

---

This article originally appeared in my blog: All Things Quality

	My name is Joe Strazzere and I'm an experienced Quality Assurance professional. I like to lead, to test, and occasionally to write about leading and testing. Find me at http://AllThingsQuality.com/.

Who Needs QA? Not Yahoo!

Yahoo! likes their developers to work without a net. 
Instead of "falling to their death", Yahoo! will now refer to this as "forcing excellence into the process".

Yahoo! has been making lots of changes to their development processes. After some small changes in their processes in 2013, a larger push from mid-2014 to the first quarter of 2015 involved getting rid of quality assurance.

As part of a program Yahoo! calls Warp Drive: a shift from batch releases of code to a system of continuous delivery, software engineers at Yahoo! are no longer permitted to hand off their completed code to another team for cross checking. Instead, the code just goes live as-is. If problems occur, it will fail and shut down systems, directly affecting Yahoo!’s customers.

Yahoo views this as a success. copyrightjoestrazzere

“Doing that,” said the company’s senior vice president of science and technology Jay Rossiter, “caused a paradigm shift in how engineers thought about problems. We said ‘No more training wheels,’ and it made a huge difference. We forced excellence into the process.”

Apparently, the work QA at Yahoo! performed added human error. “It turns out,” Rossiter said, “that when you have humans everywhere, checking this, checking that, they add so much human error into the chain that, when you take them out, even if you fail sometimes, overall you are doing better.”

A year after the change, “It’s 100 percent working,” said Amotz Maimon, the company’s chief architect. “It’s amazing. Even people who didn’t think it could ever work now think it’s great, and we are applying it to everything we do in the company.”

It's not clear if "overall you are doing better" means cheaper, faster, more error-free, or something else. And it's not clear what "applying it to everything we do in the company" means.

Many folks believe another restructuring of Yahoo!'s company is coming soon, and that the company's 10,700 employees should be worrying about their job security. Indeed, CEO Marissa Mayer has promised to announce plans for a cost-cutting reorganization late in January. So maybe getting rid of QA had something to do with cheaper, rather than better. And maybe cheaper will be applied to everything they do in the company.

It will be interesting to see if this shift to "no QA" is permanent, or just another fad. Meanwhile, we'll see if Yahoo!'s quality changes for good or for bad. Or if there's anybody left who cares.

---

This article originally appeared in my blog: All Things Quality

	My name is Joe Strazzere and I'm an experienced Quality Assurance professional. I like to lead, to test, and occasionally to write about leading and testing. Find me at http://AllThingsQuality.com/.

Poor ISTQB Foundation Certification Exam Questions

A few months ago, my team was charged with preparing for, and passing the ISTQB Foundation Level Certification Exam. I retired before the team took the exam, but was present for our preparation work.

As part of the preparation, we took a series of online "practice exams". While taking the exams, I wrote down a few questions that seemed "off" to me.

Here's one:

Test cases are designed during:

1. Test recording
2. Test planning
3. Test configuration
4. Test specification

To me, this is a poor question.

When I'm testing software, I design test cases all the time. 

• I design some test cases during a dedicated session I set aside for just that purpose. 
• Some I design and prepare while I'm initially reading the requirements (if any exist). 
• Some test cases are suggested by others after reading a draft of my list of planned tests. 
• Often, while I'm creating my test automation, other tests come to mind, and I add them to my list of tests to be execute.

What do you think? Is this a good question to ask as part of a certification exam?

By the way - what do you think is the correct answer?

---

 This article originally appeared in my blog: All Things Quality

	My name is Joe Strazzere and I'm an experienced Quality Assurance professional. I like to lead, to test, and occasionally to write about leading and testing. Find me at http://AllThingsQuality.com/.

Would You Test Software Designed To Cheat On Tests?

Das Cheat?

The U.S. Environmental Protection Administration announced Friday that the Volkswagen AG had surreptitiously equipped its diesel vehicles with software designed to recognize when those products were being tested on a dynamometer, essentially an automotive treadmill. In such a situation, the full complement of emissions controls systems would operate at their maximum, bringing the vehicles into compliance with U.S. - and even tougher California - emissions standards.

But once the testing would be completed, according to the EPA, the vehicles would revert to a different mode, effectively allowing emissions levels to increase by as much as 40 times.

• Volkswagen AG admitted that it built “defeat device” software into its diesel cars from 2009 to 2015 that automatically cheated on U.S. air-pollution tests
• Software is built into 11 million vehicles worldwide
• It could cost VW around $7.2 billion to fix the cars
• Volkswagen's CEO resigned due to the scandal
• VW designed the software to have the 3.0-liter engine run on two modes: “temperature conditioning” – the cheating mode – and “normal mode.”
• The car reverted to normal mode “exactly one second” after the software detected that EPA testing conditions had stopped
• VW has blamed a few rogue engineers for the scheme

It's pretty hard to believe "a few rogue engineers" would concoct and execute a scheme like this that, if discovered, could have catastrophic repercussions for the company. 

Imagine you worked for VW (or were a contractor), and were assigned to test this "defeat device" code. How would you feel? And how would you feel now that it's public knowledge?

See also:

http://www.npr.org/sections/thetwo-way/2015/09/22/442457697/11-million-cars-worldwide-have-emissions-problem-volkswagen-says

http://www.washingtonpost.com/news/wonkblog/wp/2015/09/22/anatomy-of-volkswagons-deception-the-recall-that-never-fixed-any-cars/

http://www.bloomberg.com/news/articles/2015-09-19/volkswagen-emissions-cheating-found-by-curious-clean-air-group

http://www.networkworld.com/article/2985059/software/so-you-re-a-vw-software-engineer.html

http://www.nbcnews.com/business/autos/volkswagen-11-million-vehicles-could-have-suspect-software-emissions-scandal-n431456

http://www.foxnews.com/leisure/2015/09/22/volkswagen-says-11-million-its-cars-are-equipped-with-emissions-test-beating/

Update:

Apparently, VW's cheating was worse than originally reported, and extends to some Audi and Porsche models as well.

The EPA, California Air Resources Board and Environment Canada discovered the additional cheating while testing Volkswagen's vehicles. They have found nothing similar while conducting tests of other car companies.

Volkswagen engineers took a sneaky approach when fitting these vehicles with the "defeat device" software, the EPA alleged.

See Also:
http://www.usatoday.com/story/money/cars/2015/11/02/epa-diesel-suv-volkswagen-audi-porsche/75044132/
http://www.nytimes.com/2015/11/03/business/some-porsche-models-found-to-have-emissions-cheating-software.html?_r=0
http://www.latimes.com/business/la-fi-vw-action-20151102-story.html
http://www.consumeraffairs.com/news/epa-more-vw-diesels-have-defeat-device-software-110215.html

---

This article originally appeared in my blog: All Things Quality

	My name is Joe Strazzere and I'm an experienced Quality Assurance professional. I like to lead, to test, and occasionally to write about leading and testing. Find me at http://AllThingsQuality.com/.

Perhaps They Should Have Tested More - SunGard

not the actual           BNY Mellon       logo *         

Perhaps SunGard and/or BNY Mellon should have tested more

If you've been watching the Market the past few weeks, you may have been concerned by all the volatility and heavy activity occurring.

And if your funds' NAV (Net Asset Value) was reported through BNY Mellon, you may have been even more concerned to learn that the NAVs at the end of the day's trading were delayed.

That's apparently because an operating system upgrade by SunGard (which hosts the fund account platform of BNY Mellon) caused a malfunction - on both the production platform, and the backup platform!

• SunGard hosts the fund account platform of BNY Mellon
• A malfunction in this platform disrupted the net asset values (NAVs) of BNY Mellon mutual funds and exchange-traded funds
• Impacted 66 of BNY Mellon's US fund accounting clients
• For up to 1200 funds valued at over $400 billion, the reporting of the NAVs at the end of the day's trading was delayed.
• The timing was especially poor, due to the heavy market activity in the days that followed.
• The problems may have led BNY Mellon to calculate inaccurate prices during the market's volatility
• BNY Mellon's clients were likely receiving imprecise information
• Per SunGard "issue appears to have been caused by an unforeseen complication resulting from an operating system change."
• SunGard said it followed standard testing procedure for the implementation of this change, but the update still corrupted both BNY Mellon's primary and backup environments
• BNY Mellon's chairman and CEO Gerald Hassell said the full restoration of SunGard's platform "has taken far longer than any of us would have expected."
• Dozens of companies rely on Bank of New York data to determine the correct value for their funds
• Bank officials say the problem began on Aug. 24, just as U.S. markets suffered their worst single-day loss since 2011

Perhaps it was just bad timing. 

It's nice that SunGard followed their standard testing procedures, but perhaps they weren't adequate. And perhaps it's not a good idea to make changes to the Backup environment, until you are completely sure the Production environment is actually working correctly.

Perhaps SunGard (and perhaps even BNY Mellon) should have tested more.

See also:

http://www.wsj.com/articles/bny-mellon-races-to-fix-pricing-glitches-before-markets-open-monday-1440960742

http://money.cnn.com/2015/08/31/investing/wall-street-glitch-bny-mellon/

http://www.bizjournals.com/philadelphia/news/2015/08/31/sungard-close-software-glitch-fix-bny-mellon.html

http://www.bizjournals.com/philadelphia/news/2015/08/28/sungard-software-glitch-bny-mellon-wall-street.html

http://www.waterstechnology.com/buy-side-technology/opinion/2423964/the-nav-crash-mess-at-bny-raises-new-questions-for-big-funds-custodians-alike

http://meredith.worldnow.com/story/29927179/bank-of-ny-glitch-leads-to-data-confusion

http://www.reuters.com/article/2015/08/31/bank-of-ny-melln-funds-idUSL4N1161RG20150831

http://www.fiercefinanceit.com/story/bny-mellon-adds-safeguards-after-sungard-glitch-lasted-longer-expected/2015-09-01

* actual BNY Mellon logo was removed per demand from BNY Mellon

"In connection to the Bank of New York Mellon Corporation’s proprietary rights over its famous trademarks we are notifying you of the following:

It has come to our attention that our trademark(s) appears as a metatag, keyword, visible or hidden text on the web site(s) located at:

http://www.allthingsquality.com/2015/09/perhaps-they-should-have-tested-more.html

without having obtained prior written authorization from The Bank of New York Mellon Corporation. This practice infringes upon our exclusive intellectual property rights."

---

This article originally appeared in my blog: All Things Quality

	My name is Joe Strazzere and I'm an experienced Quality Assurance professional. I like to lead, to test, and occasionally to write about leading and testing. Find me at http://AllThingsQuality.com/.

Perhaps They Should Have Tested More - Dilbert

http://dilbert.com/strip/2015-08-19

The "Internet of Things" is a scary place:

• New brain stimulator device had a software glitch
• Momentary homicidal mania is a side-effect
• One dead in cafeteria brawl

Perhaps Dilbert should have tested more?

---

This article originally appeared in my blog: All Things Quality

	My name is Joe Strazzere and I'm an experienced Quality Assurance professional. I like to lead, to test, and occasionally to write about leading and testing. Find me at http://AllThingsQuality.com/.

Free Web Sites Where You Can Practice Your Testing

If you miss a note, I'll whack you with this stick!

As the old joke goes: "How do you get to Carnegie Hall? Practice, Practice, Practice."

And how do you get to be a good Web Tester? Practice, Practice, Practice. copyrightjoestrazzere

Here are some free web sites that might help you with your manual or automated testing.

ParkCalc Widget

http://adam.goucher.ca/parkcalc/

copyrightjoestrazzere

A copy of the parking fee calculation widget from the Gerald R. Ford International Airport.

This "application" has some serious flaws, and has been the subject of lots of exploratory testing. Often the goal is to try and attain the highest parking fee you can, and discover the features/bugs in the widget along the way.

Read more here:

http://www.shino.de/2010/06/20/parkcalc-automation-getting-started/

http://testkeis.wordpress.com/2010/04/11/a-testing-exercise-parkcalc/

http://weekendtesting.com/archives/976

http://www.infoq.com/news/2010/05/testing_challenge

http://testingdisciple.wordpress.com/2010/07/19/my-solutions-for-parkcalc-exercise/

http://www.grr.org/ParkCalc.php

Brown and Donaldson (B&D) online brokerage Web site

http://bdonline.sqe.com/

The case study featured in The Web Testing Handbook by Steven Splaine & Stefan P. Jaskiel and the SQE Web testing training seminars.

B&D is a fictitious online brokerage firm created with the specific purpose of providing you with the opportunity to practice the Web testing techniques discussed in the book and training seminars, thereby gaining real-world interactive Web testing experience.

It even includes sample Requirements and Test Plans.

The Mercury Tours Web Site

http://newtours.demoaut.com/

Used in conjunction with the tutorials for some HP (formerly Mercury Interactive) products, such as QuickTest Pro.

Altoro Mutual Web Site

http://demo.testfire.net/

The Altoro Mutual website is published by Watchfire, Inc. for the sole purpose of demonstrating the effectiveness of Watchfire products in detecting web application vulnerabilities and website defects. This site is not a real banking site. Similarities, if any, to third party products and/or websites are purely coincidental. This site is provided "as is" without warranty of any kind, either express or implied. Watchfire does not assume any risk in relation to your use of this website. For additional Terms of Use, please go to http://www.watchfire.com/statements/terms.aspx.

InsuranceWeb Web Site

http://demo.borland.com/InsuranceWebExtJS/index.jsf

This site is a fictitious representation of an online company for the purpose of demonstrating Borland Solutions.

Find its web services documentation here:

http://demo.borland.com/InsuranceWebExtJS/webservices.jsf

Know of any other sites used for practicing your web testing craft? Shoot me a note, and I'll add them to the list and give you all the credit!

---

This article originally appeared in my blog: All Things Quality

	My name is Joe Strazzere and I'm currently a Director of Quality Assurance. I like to lead, to test, and occasionally to write about leading and testing. Find me at http://AllThingsQuality.com/.

The State Of Testing 2015

The State of Testing 2015

Back in January, I asked you to help my friend Joel Montvelisky and complete this year's survey he and his company PractiTest ran in association with the magazine Tea Time with Testers. 

Many of you responded, and the results are in!

It's an interesting read. You'll get location, background, experience, and salary information from respondents around the world.

You will find some results that probably aren't surprising ("Testers learn on their own") and some that could be surprising ("Testers are documenting less than in previous years").

You'll hear what Managers look for when hiring testers, and lots more.

Check it out - it's worth your time!

http://qablog.practitest.com/2015/07/the-state-of-testing-report-for-2015-is-out/

---

This article originally appeared in my blog: All Things Quality

	My name is Joe Strazzere and I'm an experienced Quality Assurance professional. I like to lead, to test, and occasionally to write about leading and testing. Find me at http://AllThingsQuality.com/.

Bruce Asks All Things

Ask All Things

Bruce asked a great question:

We have a large number of tests that we would like to automate.  However, it takes time to automate each test, and we have several thousand tests that could be automated. 

Is there an average or maximum number of tests that QA people automate using tools like Selenium or Ranorex?  Thanks for your help.

Sincerely, 

Bruce 

Nice question, Bruce! And one that many folks contemplate when they encounter a large system that needs a lot of test automation.

At one place I worked, we automated over 8,000 tests.

In that particular shop the UI and much of the underlying code was very, very stable. New functionality was added, but it was virtually always added in new instances, and seldom involved changes to existing instances. Thus automated regression tests were able to pay back the investment, and maintenance wasn't overly burdensome.

I know of no industry averages or maximums. Even if there were such averages, I don't see how it would be of much help for your individual case.

Instead, you should consider the full cost of automation in your shop (including initial setup and ongoing maintenance) against the benefits you expect to receive.

Some factors to consider: copyrightjoestrazzere

• Do you have sufficient expertise already on staff to create this automation, or will you need to go outside your organization to get it?
• Do you have time in your schedule to plan, design, develop, test (yes - your automation needs some testing!), document, and deploy your automation?
• How much ongoing maintenance will this automation need?
• How much is the current lack of automation costing your company - both in terms of bugs escaping to Production, and the cost of manual testing you must perform due to the lack of automation?

Many companies choose to start small. Automate only the parts that matter most (perhaps the riskiest or most critical parts of your system), weigh the costs versus benefits, and then decide what to automate next (if anything).

Some companies choose to plan automation for new systems, but only tactically go back and automate legacy systems as time permits.

Other companies choose to outsource the initial automation effort, then bring the maintenance portion in-house.

Let's throw this question to the readers - Do you know of an average or maximum number of tests? Do you know of other factors to consider as you think about automating a large system? Have you faced a large automation effort like this? If so, how did you handle it?

-joe




Do you have questions? Use the new "ASK ALL THINGS" widget over on the right-hand panel. Send me questions about anything:

• about the testing profession
• about test automation
• about bug tracking
• about being a Manager
• about testing and QA jobs
• about quality
• anything!

I'll read through the questions, pick some that not only interest you and me, but questions that I think will interest others. Together we can not only get you the answers you need, but we can provide others with some useful information as well.

Ask All Things!

---

This article originally appeared in my blog: All Things Quality

	My name is Joe Strazzere and I'm an experienced Quality Assurance professional. I like to lead, to test, and occasionally to write about leading and testing. Find me at http://AllThingsQuality.com/.