A simple method for hardware and software evaluation

Subjective data are often the only data available for making decisions and choices in both hardware and software development. These data are therefore both important and valuable. Their importance is such that it is necessary to elicit subjective data from a variety of sources and in a manner that is relatively unbiased. A method which requires participants to compare all criteria and all proposed concepts by pairs provides a non-statistical mechanism for collecting data from a broad base of expertise, for giving appropriate weight to all input, and for reducing personal bias. These characteristics make a comparison of all pairs a useful tool for hardware and software evaluation when decisions and choices must be made. The efficacy of the method was recently demonstrated in the selection of a restraint-attachment device for Space Station Freedom.