Testing as Problem Solving

Language testers are sometimes asked to say what ‘the best test’ or ‘the best testing technique’ is. In fact, there is no best test or best technique. A test which proves ideal for one purpose may be quite useless for another; a technique which may work very well in one situation can be entirely inappropriate in another. It depends on the objectives of the courses, the purpose and importance of the tests, and the resources that are available. The assumption that has to be made therefore is that each testing situation is unique and so sets a particular testing problem. In every situation the testers have to be clear about the purpose of the testing and state the testing problem as clearly as possible. Every testing problem can be expressed in the same general terms: we want to create a test or testing system which will: consistently provide accurate measures of precisely the abilities in which we are interested; have a beneficial effect on teaching; be economical in terms of time and money.

In stating the testing problem in general terms, we spoke of providing consistent measures of precisely the abilities we are interested in. A test which does this is said to be ‘valid’. The word ‘consistently’ was used in the statement of the testing problem. The consistency with which accurate measurements are made is in fact an essential ingredient of validity. If a test measures consistently, it is said to be reliable. Reliability is an absolutely essential quality of tests. All tests cost time and money – to prepare, administer, score and interpret. Time and money are in limited supply, and so there is often likely to be a conflict between what appears to be a perfect testing solution in a particular situation and considerations of practicality. To rephrase the general testing problem identified above: the basic problem is to develop tests which are valid and reliable, which have a beneficial backwash effect on teaching, and which are practical.

We have to say something about statistics. Some understanding of statistics is useful, indeed necessary, for a proper appreciation of testing matters and for successful problem solving. At the same time, we have to recognize that there is a limit to what many readers will be prepared to do, especially if they are at all afraid of mathematics. For this reason, statistical matters are kept to a minimum and are presented in terms that everyone should be able to grasp. The emphasis will be on interpretation rather than on calculation.