Testing Concepts

A test is simply a tool that is used to measure something. To narrow that definition a little – after all, we do measure stuff all the time without having any interest in testing it – a test is usually formal, in the sense that is it created and applied with a purpose and intentionally. I may measure a television set because I have an idle curiosity about its size, but if I'm in the market for a new television, and I have a specific space to put the set, then I'm measuring that TV for a very definite reason…and I'm therefore testing that television for its ability to meet my space restrictions.
The "something" that a test is measuring can often be summarized with a question:
What are the subject's characteristics or properties? This kind of measurement looks at the test subject itself.
Does the test subject pass or fail the test? This kind of measurement compares the subject, or the subjects performance or behavior, against a concrete definition of what success means. The test evaluates the subject based on that definition; if that requirement is met, the subject passes the test.
How does the subject respond to the test? This kind of measurement evaluates some arena of performance or behavior, and is usually intended to improve understanding of the test subject.
How do multiple test subjects compare in characteristics, performance or behavior? This kind of measurement creates a matrix of compared elements, allowing comparisons across various axes and data points.
A test requires more than just asking one of these questions, however. Tests must be planned and thought out a head of time; you have to decide such things as what exactly you are testing and testing for, the way the test is going to be run and applied, what steps are required, etc. A test is usually based on some kind of understanding of what a good result would be, or a specific definition of what "good" means. Using the example above, say I find a television that will fit, so I based on the fact that it passed my measurement test. I get home and set it up and then realize – oops, it doesn't come with a remote control – that I hadn't specified all of my requirements, and as a result hadn't tested for the all of the correct things that I needed.
A misunderstood or inadequately planned test can waste time and provide bad information, because the interpretation of the test results will be flawed and misleading. Oops again, I brought a metric tape measure, and I don't know how to convert. Darn, was I supposed to measure with or without the antenna? Did my wife tell me to look for a projection TV, or a Web TV?
Before running any test, you should be able to answer the following rough questions:
What are you testing? Define the test subject, whether it is a thing, a process, a behavior, a threshold, etc. Define the scope of the test subject. For example, if you are testing a web site's links, will you test every link, or only links to static pages, or only links to other pages as opposed to internal links, etc?
From what point-of-view are you testing? If your test is supposed to mimic the interaction of a specific agent or user, then you must have a strong understanding of that agent or user. [Read more about understanding users.]
What are you testing for? Be as specific as possible. If you are going to test one aspect of the test subject, make that limitation clear.
How are you going to test? Define the test plan, the specific test(s), test methodologies, etc.
What are the limits to the test? Set expectations carefully, because if the test can only measure a part of the test subject or its behavior, the results must be interpreted with this limitation in mind.
Testing, Quality Control and Quality Assurance