So, do you think our CSRs are hot?

"If we knew what it was we were doing, it would not be called research, would it?" -- Albert Einstein

Let's talk about customer satisfaction research, and in particular, drivers of customer satisfaction. This is probably one of the most sacred grounds of satisfaction analysis, and every company that offers customer sat research is usually pitching some sort of proprietary procedure or knowledge on how to get those key drivers out of the survey data. The same story refers to NPS, loyalty, or pretty much any number of more expensive measures of the same thing.

Usually, these procedures are based on some sort of correlation between overall satisfaction (NPS, loyalty) and the drivers. Some use fancier math, some use simple math, but the idea is pretty much the same. To prove that the idea is working well, your MR vendor will create a bunch of pretty charts, show you statistically significant p scores, and what not. Now, every time I see a chart that is a bit too perfect, I get a nagging feeling of suspicion - is it really happening, or we are dealing with a self fulfilling prophecy again?

Fortunately, one day I got my answer. As I mentioned before, at some point in my career I was in charge a customer sat survey, and it had one of those drivers that makes you sigh - satisfaction with store hours. So, whenever I ran the correlation between the overall sat and the drivers, I would always see that nice positive correlation with the store hours. Must be one of those important factors, right? Well, turns out, the correlation held even for the flagship stores, which were open 24 hours. I don't know how one may be dissatisfied with a 24-hour store hours, but apparently, if you piss the customers off enough, they will be. They will also think your signage colors are hideous, parking spaces are too small, and CSRs are ugly. Obviously, it is not the store hours that drive overall satisfaction, but the other way around. If anything, those bogus questions are going to correlate with overall sat very well, as they really don't reflect anything else but the overall satisfaction.

Now, every time I answer a customer sat survey (yes, I take other companys' surveys - guilty as charged), I always laugh when faced with a million dimensions, half of which I have absolutely no opinion on, except... well, they are pretty good, so I guess I am "satisfied" with the advice and information they give me.

What's the conclusion? I guess, the conclusion is that in the context of customer sat, those drivers are not of much help. There are other ways to understand what's important to your customers, and by all means you should employ them in an intelligent manner. Should your satisfaction really grow if you change those signage colors? There is a sure way to find out - change them and see if satisfaction budges. If not, move on to another variable.

Score!

"Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius -- and a lot of courage -- to move in the opposite direction." -- Albert Einstein

We have all heard of NPS, the Net Promoter Score. It is supposed to be the holly grail of loyalty, and a great alternative to your regular old tired Satisfaction Score. Maybe.

I guess it is time to share my experience and call it as I have seen it.

Numero uno - the absolute NPS score. I worked in a couple of consumer industries in marketing analytics, retail and telecommunications. If you pulled any research done... ever, you will see that retail, generally, has high satisfaction scores, and telecom - not so much. In fact, retail often gets over 60% in top two (on 10-point) or top box (on 5-point scale) . Now, if you ever look at cross shopping patterns in retail, you will see how fickle the customers are. I worked for a retail company where around 90% of its best customers cross-shopped with competitors. Yet, it had an NPS of well over 50%. I have worked for a telecom company that had an NPS way down at the bottom of the scale. I should admit that it has not always treated customers perfectly, however, customers were surprisingly loyal to their services. So, yes, it's all relative.

Numero dos - NPS and other scores. As a part of my former job I was in charge of the company's satisfaction survey. I hated the things I had to do to maintain it, but I loved the results, especially the given sample size that was well into the hundreds of thousands. Obviously, at some point it came to the NPS measurement, and as a sucker for general understanding of the nature of things, I did test that NPS against the overall satisfaction (top box on 5-point scale). Does not matter how you cut it, by weeks, by months, by stores, by regions... the NPS was 97% correlated to the Top Box. I do not know how much of the groundbreaking insight was packed in the remaining 3% of the information (OK, it's more like 6% in terms of variable variation), but I highly doubt it is going to change my view of what's going on with the customers.

So, my conclusion, basically, is that NPS is the same as good ole Satisfaction Score, freshly repackaged, and obviously, more expensive.

P.S. Next time let's talk about "drivers" of satisfaction.