By Dr. Stephen Needel
Feelings, a 1974 song by Morris Albert, might be the worst song ever to hit the charts (at least for males in the early 70s). A recent Quirk’s article (http://www.quirks.com/articles/2016/20160808.aspx) about feelings may not be the worst article they’ve ever published, but it ranks up near the top. The author is flogging a new voice analysis system that claims to detect the passion in response to a new product concept and enables better performance forecasts. Full disclosure – what I know about audio engineering and sentiment analysis would fit in a cocktail glass and leave plenty of room for my martini. The good news – you don’t need to know anything about either of these topics to appreciate the lack of empirical evidence in this article.
The article starts with, “Fundamentally, consumers adopt new products and services that improve their lives.” This is by no means fundamental; neither my new Smucker’s Chocolate Coconut Ice Cream Topping nor my new Gia Russa Bolognese Pasta Sauce is likely to improve my life, although they both taste good. Most things in our pantries are not life-altering, and food is what we all buy most of on a transactional basis.
The author believes the “enormous question on the table…is how to identify product concepts that have high probabilities of building deep emotional connections with consumers.” No, the enormous question on the table, from a new product forecasting point of view, is how to better predict new product trial; that may or may not involve emotions. “Consumers do not talk about products using multi-point scales”, the author claims, “They are unnatural modes of expression.” Consumers don’t talk much about products at all unless we ask them – they have better things to do with their lives (except those who live their lives on Facebook). A well-constructed set of scalar items is no less sensitive or less informative than the open-ended questions they would have us use. Neither modality is more or less natural to consumers.
In a comparative test, “the language-based sentiment metric [open end] yielded a coefficient of variation that was five times higher than the scalar method.” As if this is a good thing. What they obtained was a highly variable metric relative to scales, and that’s not very good for a prediction exercise. By the way, we do not expect much variation from a 5-point scale either, unless the product is particularly polarizing. And we won’t mention the issues of a COV with a non-ratio scale or the wisdom of asking 35 purchase intent questions at one time.
In lauding their voice analysis method rather than the typing-an-open-end-answer, success for the former is claimed because respondents produced 83 words per stimulus compared to the typed 14 words. I can write a scathing review of a restaurant in lots of words or I can just say “It Sucked!!!!” I’m pretty sure the latter conveys exactly how I feel about the restaurant and why you shouldn’t go there. Word counts are a ridiculous measure of anything.
There’s a lot of fake math in this article surrounding the audio sampling rate – just ignore it, it’s wrong.
Using voice analysis, the author reconfigures the maximum trial potential of a product as the subset of respondents who (a) express positive sentiment towards the concept, (b) show positive activation (an expressed desire to do something), and (c) do so in a passionate way (as defined by the voice analysis). Spoiler alert – there is no validation for this assertion. I’m not saying we, as an industry, are great at predicting new product success. I do, however, expect someone who’s claiming to have a better way to do this to show us some data to back up the claim.
I’m sure this company wants to know what I’m feeling about their new voice analysis approach. I’m not a likely adopter, because while I experienced positive activation in a passionate way, my sentiment is anything but positive.
