Lessons Learned While Testing Loosely-Coupled Systems

Hang Loose!

In my company, many of the systems we build are loosely coupled with our customers' systems, systems from our division in India, or both.

Our loosely-coupled systems have some characteristics that can make them a challenge to develop and a challenge to test:

• the interface is defined
• but partly because these systems were written long ago with no current documentation, little or no knowledge exists about how the other end uses it, nor their dependencies

For some of our systems, the interface consists of one or more XML files that are passed between systems. You might think that it would be reasonable to assume that the other end is consuming and parsing the file as XML, but you would be wrong. In one case, they were consuming and parsing the file as a flat file. The program they used to ingest the XML file in this non-standard manner was written such that order and whitespace was critical! We had to revise our testing to take this into account.

In one of our systems, a flat file was delivered via a URL as the interface. While we originally thought the customers would be consuming the file as a browser would, we found that this wasn't actually the case. The customers had written a custom program which depended on fields in the HTTP header that often changed without affecting browsers. We had to add steps in our testing to specifically inspect the header fields to make sure they never changed.

Many of our systems pass flat files back and forth via FTP. When we test these systems, we must consider:

• What if the file arrives early?
• What if the file arrives late?
• What if the file never arrives?
• What if the file is corrupt?
• What if the file is read-only?
• What if the file is empty?
• What if the file is much bigger than expected?
• What if more than one file arrives?
• What if you try to use the file while it is being written, but is not yet complete?
• We often use a ".Done" file which is created only after the primary file is ready to be picked up, or indicate "done-ness" in some other way
• What if the file contains a virus?

Lessons: copyrightjoestrazzere

• Don't assume!
• Document as many of the interface rules as you can
• Define the interface as precisely as you can
• There must be lots of defensive programming, which must be tested
• You must plan to prepare and execute many negative cases
• In future versions, prepare to detect ANY changes in the interface, not matter how trivial, no matter how seemingly-insignificant

---

This article originally appeared in my blog: All Things Quality

	My name is Joe Strazzere and I'm currently a Director of Quality Assurance. I like to lead, to test, and occasionally to write about leading and testing. Find me at http://AllThingsQuality.com/.