February 28, 2006

Priority and Severity

I always prefer to keep things simple.

When tracking bugs, for example, I tend to combine Priority and Severity into one field (usually called Priority), and have only three to five settings.
Here's what we use now:
  • Critical
  • High
  • Medium
  • Low
Everywhere I have worked, there have been requests for more levels. But I've never seen a compelling reason for more than five. copyrightjoestrazzere

When people insist on more and more levels, I like to send them this dialog from the movie "This is Spinal Tap":
Nigel Tufnel: The numbers all go to eleven. Look, right across the board, eleven, eleven, eleven and...

Marty DiBergi: Oh, I see. And most amps go up to ten?

Nigel: Exactly.

Marty: Does that mean it's louder? Is it any louder?

Nigel: Well, it's one louder, isn't it? It's not ten. You see, most blokes, you know, will be playing at ten. You're on ten here, all the way up, all the way up, all the way up, you're on ten on your guitar. Where can you go from there? Where?

Marty: I don't know.

Nigel: Nowhere. Exactly. What we do is, if we need that extra push over the cliff, you know what we do?

Marty: Put it up to eleven.

Nigel: Eleven. Exactly. One louder.

Marty: Why don't you just make ten louder and make ten be the top number and make that a little louder?

Nigel: [pause] These go to eleven.
Labels:

Automation-Assisted Test Fest

On occasion, I have conducted some "automation-assisted manual testing" where our team of developers and QAers performed scripted manual testing while our automated tools simultaneously generated a load on the system. copyrightjoestrazzere

It was very useful to have people actually seeing the system as it was slowing down (and sometimes failing).

We would usully hold a 1-hour "test fest" followed by a 1/2 hour discussion about what we saw.

I provided 10-20 written scenarios for people to follow during the session. "Festers" were asked to accomplish specific tasks and note any problems they observed, or unexpected slowdowns, along with the time the problems occurred.

During the followup, we talked about what we observed and I collected their notes. I used their input to correlate the problems found with the load on the system, and to write bug reports as needed.

It proved to be a rather useful, inexpensive, fun exercise which could flush out potential problems and give everyone more confidence in the system-under-test.
Labels:

February 27, 2006

Perhaps They Should Have Tested More - Minneapolis School District

Contributed by my friend JakeBrake.


To me, the most interesting parts were:
  • Now, the district says its down to a two percent error rate.
  • "We're committed to gaining the trust back..."

Two percent error rate in paychecks? So, every payroll period there are 140 incorrect checks? That doesn't sound very trustworthy to me.


KSTP.com - 5 EYEWITNESS NEWS

Teachers outraged over paycheck problems


Updated: 09/26/2006 10:32:35 PM -
"People are missing their mortgage payments, they are having to take out credit cards in order to pay for school supplies," said teacher Megan Marsnik.


"We're committed to gaining the trust back as I mentioned one payroll at a time to get better and better," said Dan Loewenson, of the Minneapolis School District.

Imagine, it's payday, but your check never arrives. Or, that check is for hundreds less than its supposed to be.


A computer glitch has left thousands of Minneapolis teachers unpaid, and Tuesday night, they let the school board know how upset they are.

"I am a new teacher and I don't make anywhere near $6,000 per pay period," Roosevelt Teacher Brian Nutter told the school board. "But to have $6,000 subtracted in one pay period was a nightmare."

Many paychecks were short or inaccurate, and a few didn't show up at all.


"People are missing their mortgage payments, they are having to take out credit cards in order to pay for school supplies, my kids couldn't even get school clothes this year and I'm a teacher," said teacher Megan Marsnik.


The problems started when the district switched to a new payroll software this year. Human and technical errors caused glitches. At one point five percent of the districts 7,000 payroll checks were wrong.


Now, the district says its down to a two percent error rate.


"We're committed to gaining the trust back as I mentioned one payroll at a time to get better and better," said Dan Loewenson, of the Minneapolis School District.


But some teachers say its too little, too late, because it's hard enough to make it on their rightful salary. Inconsistent checks, they say, add to an already stressful environment.


"It's affecting all of us and what we want is to get regular paychecks, it's not a lot to ask," Marsnik said.
Labels: ,

February 24, 2006

My QA Magazine Rack

Here are some magazines I have on my shelf:
Labels:

Organizing a Test Fest

In the past, I've organized some testing sessions for non-QAers.  We called it a "Test Fest". (fun, fun, fun!)

It usually turned out to be a very valuable exercise.

In order to get buy-in:
  • Advertise. Make it sound like an exciting opportunity to help the product be better with everyone's input.
  • Plan a recap session immediately after the test. During this recap - solicit feedback about the user experiences, but most of all thank everyone publicly!
  • Food always helps. copyrightjoestrazzere
  • Sometimes prizes (the best bug?) can help. Nothing elaborate - a box of chocolates, a bottle of wine, something like that. Winning a prize is more important than the value of the prize itself.
  • Make sure that people know what they are getting into, how long it will take, and what they will be expected to do. Many people can carve out an hour or two from their schedule if they know far enough in advance.
  • Make sure that people know that you will give them the details of what they will be doing. Some folks may be worried that they will look stupid if they don't know what to do. Don't let them be worried.
  • In my sessions, I had 5 or 6 exercises to be carried out, in any order the tester chose. I handed out sheets with all the instructions needed, screenshots, etc, along with any data needed. Each exercise was on one page, with space to write down any unexpected results.
  • Make sure management is committted. You don't want folks to worry that their managers would rather that they didn't participate.
  • Have assigned "helpers" - people who can run over to a tester who needs immediate help, so that they aren't blocked for any length of time.
  • Make sure that users note the time when they start and end a task, and when anything unexpected occurs. This can help correlate their observations to messages you find in the logs.
Labels:

My QA Bookshelf

Here are some of the books currently in my bookcase:
(last updated September 4, 2009)
Labels: ,

Tester's Tool Box


Here are some great tools you may want in your Tester's Tool Box:


AllPairsAllPairs is a free tool - written by James Bach - which is useful for generating test pairs for testing combinations of software features, two at a time.
BareGrepFree file finding and text searching tool.
BareTailFree real-time log file monitoring and highlighting tool.
BlatWin32 Command Line e-mailing tool.
BugzillaThe famous, free, open-source defect tracking system.
DaphneDaphne is a free system tray application for killing, controlling and debugging Windows processes.
DbVisualizerFree cross-platform database independent visual browsing and editing tool.
EtherealTool for capturing and analyzing network traffic.
FileZillaOpen Source FTP client and Server.
firstobject XML EditFree tool for editing very large XML files.
HttpWatchHTTP viewer and debugger.
InstallWatchRecords modifications made to your PC during the installation of software, hardware, or configuration changes.
Internet Explorer Developer ToolbarThe IE Developer Toolbar provides several features for exploring and understanding Web pages.
IrfanViewA small freeware graphic viewer for Windows.
JennyJenny is tool for generating pairwise regression tests.
JR Screen RulerFree virtual ruler for your computer screen.
Log-WatchTool - written by Janes Bach - to watch a log file and play a sound when the desired text appears.
LogParserTool that provides universal query access to text-based data such as log files, XML files and CSV files, as well as key data sources on the Windows operating system such as the Event Log, the Registry, the file system, and Active Directory.
MyIVOFree remote PC access tool.
OpenSTAOpenSTA is an open source tool used for HTTP and HTTPS load and performance testing.
PerlClipPerlClip is a free tool - written by James Bach - used to create strings of test data.
QlockQlock is a free World Clock for both your browser and your desktop.
Screen Rip 32A freeware screen capture utility that let's you capture areas of the screen with a variety of methods.
SmartReplaceFree tool to search and replace text and file names at once in one simple action.
StatOwlComprehensive Aggregate Internet Usage Statistics.
SysInternals UtilitiesA collection of useful and powerful utilities for Windows.
TimeSnapper ClassicAutomatic screenshot journal tool.
Virtual PCVirtual PC lets you create separate virtual machines on your Windows desktop.
Xenu LinkSleuthSite spidering tool used to check for broken links.
XML NotepadFree tool for viewing, editing and comparing XML files.
WebMonFreeware web page update monitoring program by Colin Markwell.
WinMergeOpen Source visual text file differencing and merging tool for Win32 platforms.
WinTaskA terrific general-purpose Windows and browser automation tool.

(last updated June 2, 2009)
Labels: ,

February 1, 2006

My Football Bookshelf

I'm a big fan of NFL Football in general, and the New England Patriots in particular.
Here are a few books I have on my bookshelf:
Labels: ,
Subscribe to: Posts (Atom)