I’m a creative, so I’m well used to failure. Since the era of Mad Men ‘3 ideas’ has been the requisite portion demanded by suits on both agency and client side. Between them, they then tend to pick the weakest of the three (i’m only kidding lusm!) and the other two get ditched. Every single time, two thirds of my work goes in the bin. It gives you a tough skin.
In the long-distant olden days, ooh about five or ten years ago, when we were still thrashing around in the shallows of t’interweb, we relied entirely on subjectivity to decide.
“That’s a terrific shot of a swan you’ve used, I’ll buy it” (real feedback)
“The client doesn’t like dogs, they think dogs are offensive in mainland Europe” (that one was real too. I know…)
That kind of wet-finger-in-the-air feedback was, at once, frustrating and easy to dismiss. Now, we face a new breed of ‘pseudobjectivity’. The ‘numbers never lie’ feedback. Now, we all know that numbers lie. They lie for fun. Anyone with half an ounce of statistical knowledge will know that goalposts can be moved to suit the required outcome. But sometimes, statistical feedback is given without any goalposts at all. Not even a corner flag.
“It didn’t resonate with our target audience, they just didn’t click like they used to’. Now, whether that is from a focus group or from real customers, it sounds pretty damning. Objective feedback for once. They didn’t click through in the same numbers. End of story.
Focus groups have their value. They give insight and depth to research, for example. But they are notoriously bad at picking great creative work. One focus group famously said that ‘brown’ would be the worst colour for a soft drink. It does sound yucky, doesn’t it? (come on, keep up at the back)
The team behind the Cadbury’s gorilla had to put many, many iterations into research until they found the right ‘formula’ for that ad. For ‘formula’ I reckon you can substitute ‘focus group’. Finally, a group ‘got it’.
And as for real customers, so only four people clicked through? Did they then buy your product? Is that maybe better than the dozen or so lazy browsers? Was it the creative execution or was it perhaps the content? Have I just polished the turgid? Do people perhaps just not want another chocolate fireguard? In summer. Are we aiming this at your warm database or cold customers? Did the last piece of work carry different images? Was it more socially enabled, was it a better deal, what was the weather like? So many possible reasons, yet the blame falls squarely on the creative work. As it always did. Not the strategy. Rarely the media. Never the product. My fault, I guess, for not nailing exactly ‘what success looks like’. Lesson learned.
Now I don’t want to lose all my strategic numerate friends. Many of them do an incredible job and dig far deeper and more intelligently into data than I will ever comprehend. If you are not one of these brilliant minds, if you are just rattling the abacus for effect, please go back to feedback on a ‘feeling’. For an enthusiastic amateur, claiming failure by numbers is statistical correctness gone mad.