Abstract:

 

In product tests and surveys, bias occurs when a measure from a sample systematically differs from the population measure of interest. To put this into statistical terms, a statistic is biased if it systematically deviates from a population parameter, irrespective of the sample size. The existence of bias determines whether one needs a control product or item in a product test or survey.

 

There are numerous sources of bias that have been identified. These include sampling bias, where a sample of participants or the items to be tested do not represent either the target population or the real test items. Bias also may occur when participation or non-response in a survey is not random, so that the opinions expressed do not represent the target population. Leading questions, interviewer effects, and uncontrolled individual differences can all contribute to bias. Position bias and code bias are two sources that will be discussed in this report. These two sources are relatively easy to control but code bias, in particular, is often ignored in practice. In the case of code bias, the codes themselves may contribute to the responses selected. An extreme example that we have observed involved data from a Chinese research supplier. The Chinese ideogram for the number 4 is close in appearance to that for “death” and therefore the number “4” is often avoided in practical situations such as a floor number in buildings and hotels. When one of the products in their study was coded as “444”, it was not surprising that it received a poor hedonic rating. There are many other less dramatic sources of code bias that, if not controlled, may lead to inaccurate parameter estimates.

​

​

​