What is Mystery Shopper Interview Bias?

Aug 26, 2009 by Elaine Buxton

Businesses considering mystery shopping services sometimes ask us about “interview bias,” or the concern that the mystery shopper will sway the employee to behave in a certain way or that the mystery shopper will bring opinions to the shop visit which might affect the outcome of the research.

Mystery shopping requires human interactions. Perfection is not possible. However, steps can be taken to ensure that interviewer bias is minimized so that mystery shopping value is maximized.

Decide what to test. Mystery shopping is performance research. In order to compare how each location is performing, it is important to decide what that performance should be. It is impossible to ask a mystery shopper to test performance without telling the shopper what the test is. Thus, the all-important first step is to determine what can and should be tested. Without requiring that each shopper approach the shop the same way, the results will be a hodge-podge of random events, reported on a meaningless survey form.
Here are some examples of how random experiences can be categorized into “tests” so that mystery shop results can be reasonably compared:

Banking. Instead of sending in mystery shoppers to inquire about opening “an account”, send in shoppers to ask about opening the same type of account. This allows comparisons of service descriptions when shoppers present the same scenario.
Restaurants. Send mystery shoppers during specific meal time windows. Units are more likely to be staffed similar, selling similar product and enjoying about the same amount of customer traffic.
Retail stores. Send mystery shoppers to look for specific product categories or departments.
Automotive services. Send mystery shoppers for the same type of vehicle service.

Inform mystery shoppers of required observations, not expectations. Mystery shoppers should be given enough information to conduct their visits and gather the appropriate information and observations without biasing them with the client goals, expected outcomes, etc. Granted, the required observations and survey questions will, to some extent, provide the mystery shoppers with a clue about what is considered to be acceptable. However, a mystery shopping program that over-educates the mystery shoppers will assure interview bias. Our firm has found that keeping the scoring invisible to the mystery shoppers reduces bias. Telling the mystery shopper that a question weighs heavily in the program will serve to discourage the mystery shopper from reporting a negative observation.

Select mystery shoppers based on their observation skills rather than on critical skills. A mystery shoppers’ ability to objectively observe and report items required on the shop survey is key. Over time, a shopper’s ability to remain unbiased will wane. Our firm rotates shoppers on projects to avoid this bias. Quality assurance monitoring will also reveal when opinions start to creep into a shopper’s report commentary (verbatims). This quality assurance effort serves as a constant procedure for monitoring and removing biased shoppers.

Don’t use mystery shopping to measure customer perceptions. Mystery shopping measures performance. Customer satisfaction measures perceptions. Know the difference and how to use the information wisely. For a refresher on these differences, see our blog entry “Customer Satisfaction Research vs. Mystery Shopping: A Dialogue Revisited”.

Mystery shopper interview bias is expected. A well-planned mystery shopping program will minimize the bias while maximizing the program value.

Tagged: mystery shopping, mystery shopper, customer satisfaction, casual dining mystery shopping, bank customer service, bank mystery shopping, branch mystery shop, automotive services mystery shopping, financial services mystery shopping, retail mystery shopping