Asking the Right Questions

This is a great follow up to the previous post about choice and product design. This video features Malcolm Gladwell, a social psychologist whose work has contributed to a plethora of subjects (teachers in 4 of my classes assigned readings from his works) and who was among Time’s 100 most influential people.


The main subject of his talk is how Howard Moskowitz reinvented product development in the food industry. Moskowitz had been hired to determine what the ideal level of sugar a new Pepsi drink was supposed to have (somewhere between 8-12%). What one would ordinarily do is make a bunch of different batches with different levels of sugar, increasing in small increments, and survey a large sample of people to find the 'ideal' level of sugar. However, this experiment seemed to produce no statistically significant result for Moskowitz. This led him to realize that he was asking the survey participants the wrong question- “He was asking for what was the best Pepsi; what he should have been asking was what are the best Pepsis?” In other words, there was not one ideal Pepsi that people were looking for, but multiple optimal solutions.

This realization led him to make breakthroughs in other food categories. When trying to find a superior spaghetti sauce for Ragu, he changed his approach. Instead of changing a limited set of variables in the batches, he changed every variable the he could think of, including how coarse the sauce was. A third of participants showed preference for a 'chunky' textured sauce; this proved to be revolutionary, as it contradicted the conventional notion of 'fine, authentic Italian sauces' and was not a preference that people would think of mentioning when being surveyed (especially since they have a bias towards the idealized traditional sauce). This opened up a third of the established market to Ragu and made them millions.

My takeaways from Gladwell's story are the following

1. The wording of your questions are crucial. People were giving Moskowitz the right answers to his question- he was just asking the wrong one! When we get a lot of ‘noise’ in our survey results the immediate response should be to analyze the survey and not to pick an average value in the hope of satisfying the middle of the bell curve.
2. How you ask the question matters. When you survey people, they will not always say what they really mean to say. Gladwell compares this to asking someone what their ideal coffee is; they will immediately conjure up the picture of a ‘nice dark hearty roast’ even if they like their coffee ‘milky and cold.’ Some creative experimentation like the spaghetti sauce experiment may be required to get the information that you want from your customers.