A mystery shopping report validation checklist is what separates a client who trusts every number on the dashboard from a client who calls about the one report that said the store was closed on a Tuesday. The validator sits between the shopper and the client, and the 15 checks below catch most of what goes wrong. The example throughout is a wave of 120 visits where the client should see each report four days after the visit: two days for the shopper to fill in, two for the validator.
What validation is for
Clients see validated reports only. In Shopper-X the client's report list has one row per completed and validated visit. Submitted, returned and unacceptable reports do not appear. "Validated" is the moment a report becomes a fact the client will act on.
The validator role is narrow on purpose. It has the report queue, the report editor with every answer and file, the score, accept, return for corrections, client remarks and export. It does not manage the schedule, the questionnaires, the shoppers or the payments, so a part-time reviewer can take 120 reports without access to the rest of the system. The report validation page describes the role.
The queue is organized per customer, wave and questionnaire, with three counts at the top: received, validated, not validated. On day three of the example wave they might read 74 received, 31 validated, 43 not validated. The last number is the day's work. Take it oldest first, so the two-day target holds for every report, not just the average.
The 15 checks
Run them in this order: a report that fails completeness goes back before you spend ten minutes on consistency.
Completeness
- 1. Every required answer is present. The form refuses a submission with an unanswered required question, so this rarely fails outright. Look for the answer that passes the rule and misses the point: "n/a" in a text field, or the middle option on every scale.
- 2. Every required photo is attached. Two required photos means two different photos, not the same frame uploaded twice.
- 3. Comments meet the minimum length. The editor shows a character count per text answer and labels it "Enough symbols" when the minimum is met. Then read it: eighty characters of "everything was fine, nothing to add" clears the counter and says nothing.
- 4. A comment exists where the maximum result was given. A perfect score always comes with a reason; if the questionnaire does not enforce it, you do.
Consistency
- 5. Receipt time matches the reported time. Receipt printed at 14:07, time of check entered as 15:30: ask. One of the two is wrong.
- 6. Yes/no answers match the narrative. "Were you greeted within 30 seconds? Yes" next to "I waited at the counter for several minutes before anyone noticed me" is the most common inconsistency.
- 7. Scores match the comments. Five out of five for cleanliness with a comment about a sticky floor near the till. One of them has to change, and only the shopper knows which.
- 8. The visit is inside the audit period and the time window. A visit on the right day at 9:10 when the window was 12:00 to 18:00 is not the visit the client paid for.
Scenario compliance
- 9. The persona was followed. The brief said to ask about a family membership; the report describes a student discount conversation.
- 10. The purchase was made. Where the scenario requires a purchase, there is a receipt for roughly the amount the agency reimburses.
- 11. Deviations are explained. Store closed for stocktaking, product out of stock. An explained deviation can still be a valid report; an unexplained one, where the shopper filled in the rest anyway, cannot.
Evidence
- 12. Photos show what the answer claims. "Signage visible: yes" needs a photo with the signage in frame, not a blurred corner of a door. The post on photo evidence covers how to specify photos so this check passes more often.
- 13. The receipt is legible. Store, date, time and total readable without zooming. A receipt you cannot read is no proof.
Tone and privacy
- 14. Facts, not opinions. "The assistant seemed bored" is an opinion the store manager will argue with. "The assistant did not look up from the phone for the first 40 seconds" is a fact the manager can act on.
- 15. No staff names or private data where the brief forbids them. Name badges in the text, number plates and other customers' faces in photos. Questions that collect sensitive content can be flagged in the questionnaire as containing private data, so you know where to look.
Three outcomes
Accept. Save the validator score (next section) and mark the report as validated. From that moment it appears in the client's reports and feeds the statistics.
Return for corrections. Say a report in the example wave is missing two photos: the receipt and the promotional stand. Put a remark on each question with the remark pencil ("photo of the receipt is missing", "photo of the promo stand by the entrance is missing") and return the report. The shopper gets an Edit button on that audit in the inbox and in the mobile app, adds the two photos and resubmits. The audit shows as returned until the report is back.
Unacceptable. For a report that cannot be repaired: scenario not followed, wrong location, wrong day, invented content. It leaves the queue for the unacceptable tab, the client never sees it, and the agency leaves it out of the shopper's payment.
A client remark is separate from the outcome: saving one emails the text to the shopper, who learns what is wrong without logging in.
Score the shopper while you are in the report
Score the shopper while the evidence is in front of you. A week later you will not remember which report had the blurred receipt.
Shopper-X scores each validated report on four criteria: scenario, report, evidence and timeliness, with default weights of 40, 30, 20 and 10, adjustable per questionnaire. The average over a shopper's reports sets the rating tier: Bronze below 70, Silver from 70 to 90, Gold at 90 and above. Gold shoppers are offered work first.
Example: a report that is perfect on scenario, content and evidence but arrived a day after the fill-in deadline scores 100 on the first three criteria and 0 on timeliness. That is 40 + 30 + 20 + 0 = 90, still at the Gold threshold. A late but excellent report should cost something; it should not cost everything. The post on mystery shopping KPIs explains how the rating and the program metrics fit together.
Working as a team
Two validators splitting 120 reports need four rules.
Split by questionnaire or location set, not by day. One reviewer who sees all 30 reports from a scenario notices that every shopper misread the same question; two reviewers seeing 15 each would not.
Rely on the report lock. Two reviewers cannot edit the same report at once. The lock shows who is inside and expires on its own; the unlock action releases a report someone left open.
Re-validate when you need to. A validator's results can be unlinked so another reviewer validates those reports again. Use it for a new validator's first batch or when a client challenges a set of reports.
Trust the log. Every status change, score and remark is logged, which settles it when a shopper says the report was returned twice for the same thing.
Turnaround targets and what they tell you
| Metric | Example wave (120 reports) | What a bad number means |
|---|---|---|
| Validated within 48 hours of submission | 102 of 120 (85%) | Validator capacity, or the queue is not worked oldest first |
| Returned rate | 21 of 120 (17.5%) | Brief or questionnaire problem |
| Unacceptable rate | 3 of 120 (2.5%) | Shopper problem if concentrated, brief problem if spread |
A returned rate above 15% in one wave is almost never a shopper problem. Look at which check caused the returns. If 14 of the 21 returns are the same missing photo, the photo requirement on that question is unclear or the briefing never showed what the photo should look like. Fix the question, not the shoppers. Unacceptable reports concentrated on one or two shoppers are a shopper problem; spread across many, they point at the scenario.
The four-day target is two days of fill-in time set for the wave in the schedule settings, plus two days of validation counted from submission. If the 48-hour share drops below about 80%, add a validator for the peak days rather than extending the target.
Wording help for remarks
A remark written on the fortieth report of the day reads harsher than intended. A validator in Shopper-X can send the draft remark to the AI text improvement service and get suggested wording back. Keep the facts, soften the tone.
Draft: "You again forgot the receipt photo. Resubmit." Suggested: "The receipt photo is missing from question 12. Please add it and resubmit the report." Read it before sending; the specific detail (which question, which photo) must stay in.
Printable checklist
- Every required answer is present and says something.
- Every required photo is attached, and they are different photos.
- Comments meet the minimum length and carry content.
- A comment exists where the maximum result was given.
- Receipt time matches the reported time of check.
- Yes/no answers match the narrative.
- Scores match the comments.
- Visit is inside the audit period and the time window.
- The persona in the brief was followed.
- The required purchase was made and the receipt is attached.
- Any deviation from the scenario is explained.
- Photos show what the answer claims.
- The receipt is legible.
- The text states facts, not opinions.
- No staff names or private data where the brief forbids them.
FAQ
How many times can a report be returned?
The return-and-resubmit loop repeats until the report is validated or marked unacceptable. Set a house rule and tell shoppers: two rounds of corrections, then unacceptable. Specific remarks resolve most returns in one round.
Who sees internal notes?
Internal notes are a head field of the admin and validator editor, the part of the system the client role does not have. Administrators and validators read them; keep anything meant for the client out of them.
Is an unacceptable report paid?
Normally no. The fee stays with the audit, so the agency can leave an unacceptable audit out of the payment period, or reduce the fee in the report head when the client still gets partial value. The post on how to pay mystery shoppers covers fees, reductions and payment periods.
To talk through your own validation rules and turnaround targets, contact us.