The past few years have seen a rise in discourse on how Mystery Shopping, as a means of evaluating customer service, is losing its relevance, and there are even some who foretell that it will disappear entirely.   

As novel methods of data collection and new approaches to evaluation of customer experience have emerged, some companies reduced their programs or dismissed Mystery Shopping fully. However, let’s see what has changed about the industry, what approaches to research have come into existence and what kind of influence it all has had.    

Sensory devices

In Russia, the practice of evaluating service quality with sensory devices has been an actively developing trend for the last 5 or 6 years. This is achieved by mounting a recording device at the workplace of a target employee, to document customer interactions and transfer audio records to the server through wi-fi 24/7. (Video recording is also possible but may have legal limitations on the handling of data). The recordings can be processed by either authorized employees or AI, then checklists on consultation performance and adherence to set standards are filled in. 

Advantages:

1. A wide reach with minimal costs - Russia is known for long distances between locations, and mystery shoppers are not always available in smaller towns. In such situations they often travel over from the nearest large cities, with all related transport costs compensated, or else the frequency of checks is reduced. Putting a recording device in service therefore eliminates the necessity of searching for a new shopper each time a check is due.


2. Accumulation of bigger data volumes - As sensory devices can record service interactions day and night, it makes it possible to collect a larger volume of data, which cannot be achieved through face-to-face (f2f) visits.


3. Rude or disputable interactions are revealed - Now and then it happens that employees are rude in the course of a consultation or fail to provide customers with accurate information which results in a customer complaint. In such cases the audio recordings can be checked and verified to see whether the employee was acting properly and to make the right decision for dealing with the situation.

Disadvantages:

1. Quality of recordings - The sound quality depends on many variable factors, such as the placement of the device, the background noises in the room, the volume of the speaker’s voice. Sometimes employees deliberately hamper the successful recording process or inadvertently block the microphone or disconnect the device from the network.


2. Confidentiality and data privacy - In banking, some client data may be personal or contain information that cannot be divulged to third parties. If such data is leaked, it can cause serious damage both to the clients and the company.


3. Exclusive workplaces - If an employee isn’t stationary and moves all the time, a stationary device is not a practical option. For such situations there are smaller, badge-like devices, however they also have their disadvantages – they require charging and constant maintenance, and it’s not recommended to wear them under one’s clothes (the sound quality is drastically impaired).


4. Wi-fi dependency - Data transfer requires a stable Internet connection, as large data volumes are being sent. It can be an issue at a highway gas station, for instance, which limits the applicability of the method.

Crowdsourcing

Some companies reach out to their clients to evaluate service quality and audit their own standards. In Russia, there are examples of major retail chains, gas station chains and airlines, introducing their own Mystery Shopping mobile apps. Customers are able to complete a small task at a store (take pictures of the shelves, windows, expired products) and gain bonuses to spend later. 

Advantages:

1. Low reward-related costs - Instead of real money, certain companies have a bonus reward system for their shoppers (sometimes there are personalized discounts), where the bonuses can be spent only to purchase either products or services of such companies. This may be a motive for the shoppers to continue using the services of the companies, like ordinary customers. The companies can minimize their costs, because bonuses are not equivalent to actual money. Additionally, the customers don't always spend their bonuses immediately or may even fail to put them to use at all.


2. Speedy recruitment - If a company has a large customer base, it’s quite easy to find enough shoppers for a check. The communication here is simplified to sending push-notifications to a chosen group of customers, which takes minimum time.

Disadvantages:

1. Fraud - The ease of picking up a task can push employees into doing checks on their own and faking the results, because only a few companies validate their shoppers’ profiles. The customers themselves may even be able to attach data gathered during previous checks, or provide inaccurate information, if they forget to do something. This can result in a biased evaluation with no proper control.


2. Complex scenarios are impossible - Crowdsourcing works well when simple tasks are involved - to take a few pictures, to take down a case of standard violations or to ask staff some easy questions. However, if the requirement is to check product knowledge or services, or to memorize a lot of information at once, the effectiveness of data collection is seriously diminished as ordinary customers don’t have the skills to conduct such checks and it’s hard to teach them all the specifics of a complex task without individual training.


3. Failure to keep to the research method - When projects are given to departments that don’t specialize in them and have no relevant skills or expertise, they may make errors in the research method used. For instance, a questionnaire can be written without taking into account a variety of situations, without a logical progression, or without a detailed guide for shoppers. Important stages of research may be missing, such as the validation of submitted questionnaires. As a result, the quality of the data is low and there’s no practical use in them.

Collection of customer feedback

With the development of social media, e-cards and marketplaces, there’s been an increase of the number of services that collect and analyze customer feedback and scores on certain services, products or sales chains.

Advantages:

1. A lot of information is collected within a short time - The main advantages are speedy processing of data and the quantity of the information it yields. Modern technologies allow aggregating comments and reviews from various sources, analyzing their emotional colouring and revealing the points of customer satisfaction or stress within mere days.  


2. An opportunity to conduct competitive analysis - It’s also possible to make a quick comparison vs. competitors to understand which of them a company or a product are compared with and what the higher- and lower-scoring points are. All that takes very little time, and there’s no need for separate research.


3. An opportunity to see how employees handle feedback - If customers review sales point on maps, it’s possible to check whether the employees respond to complaints and issues, if they are quick with the follow-up, if the problems are handled successfully, toning down the negative feelings. These criteria can be turned into KPIs for each sales point.

Disadvantages:

1. Biased evaluation - Customers can exaggerate on a situation or describe it in a highly emotional way, which gives no opportunity to evaluate it objectively. For instance, one client can be unhappy with the lack of response to their phone call at non-working hours (which they can forget to mention), and the other will insist that it took a while to get a response, even though they have waited for less than a half of a minute. Such comments reveal the points to work at, but it’s a must to gauge everything well before making a decision, to avoid hasty actions.


2. The prevalence of negative reviews - It’s more probable that a customer will leave a negative review after an unsatisfactory experience of dealing with a company, than describe a positive interaction. As a result, we can get a one-sided idea of the service and fail to understand what makes customers value the company. 


3. Impossible to evaluate all points of contact - Customers mostly provide feedback on the matters that seem most important or problematic to them. It helps getting all the pieces of the puzzle, but we may not be seeing the whole picture. Many contact points are left out, and it takes additional research to evaluate them.

Summary

New methods and approaches provide a many-sided evaluation of client-company interactions, as well as of company services, and allow revealing pressure points. Collecting such data requires less time and yields more information, as compared to the classic Mystery Shopping.

Does it mean that Mystery Shopping is no longer the thing? – No! It’s still the only way to evaluate the service standards of a chain plus the employee behavior in various situations and to discover systemic issues at different stages of service provision. Data methods, collected through different studies, can complement each other and give a better idea of what’s being researched, but using only one method may result in a wrong interpretation of results and, hence, incorrect managerial solutions.

by Valeriy Khrapov, Head of the MS Departament at Romir