Erika Hall has a great write-up about the danger of using surveys, especially as a replacement to user research.
She hits many of the main problems I’ve experienced with surveys: people are not good at reporting what they like or don’t like, people can’t accurately predict their future behaviour, and not everything can be measured on a rating scale. To truly understand what customers are doing and their preferences you have to (gasp!) sit down and observe them.
Customer satisfaction is useless
The article also discusses the issues with tracking customer satisfaction scores. Erika suggests that businesses should be looking at more specific metrics that are tied to habits:
I want everyone to see customer loyalty for what it is — habit. And to be more successful creating loyalty, you need to measure the things that build habit.
The business world needs more examples of the critical numbers that measure habit, but one that comes to mind is Slack’s use of 2,000 messages as their critical number:
“Based on experience of which companies stuck with us and which didn’t, we decided that any team that has exchanged 2,000 messages in its history has tried Slack — really tried it,” Butterfield says. “It hit us that, regardless of any other factor, after 2,000 messages, 93% of those customers are still using Slack today.”
As soon as you have those numbers enshrined at your company, you can start working on innovative ways to move people toward those milestones — whether it’s email reminders, or prompting them to take new actions in the product. Because Slack knows 2,000 is its golden number, it can iterate on ways to get customers across that line.
This is a great example of how a specific customer habit can drive better decisions. User research comes in many forms, but the best research is specific, meaningful, and tied directly to what people are doing.