Posts Tagged marketing
Business community discovers that “Experimentation Works”
Posted by mark in design of experiments on April 21, 2020
Last month the Wall Street Journal “Bookshelf” (3/15/20, David A. Shaywitz) featured a review of a book about The Surprising Power of Business Experiments.
“Tests at Microsoft in 2012 revealed that a tiny adjustment in the way its Bing search engine displayed ad headlines resulted in a 12% increase in revenue, translating into an extra $100 million annually for the company in the U.S. alone.”
Stefan Thomke, author of Experimentation Works: The Surprising Power of Business Experiments.
It’s great to see attention paid to the huge advantages gained from statistically rigorous experiments. However, vastly greater returns await those willing to go beyond simple-comparative one-factor A/B testing to multifactor design of experiments. The reason is obvious: Only by testing more than one factor at a time, can interactions be discovered.
A case in point is provided by an experiment I did on postcard advertisements. It produced a non-intuitive finding that, unlike marketers, our engineering clients preferred less colorful layouts. Knowing this, we succeeded in increasing our response at a far lower printing cost. See the proof in the interaction plot at the conclusion of this white paper on That Voodoo We Do – Marketers Are Embracing Statistical Design of Experiments.
Another compelling example of the value of multifactor testing is illustrated by website-conversion results* shown here—produced from a replicated, full, two-level factorial design.
The key to a more than 5-fold increase in clicks turned out to be the combination of going to a modern font (factor A) with a more compelling button label (C). A third factor (B), background being white versus blue, did not create a significant effect, which also provided valuable insights on the drivers for conversion.
Why settle for testing only one factor when, without investing much more time, if any, you can investigate many factors and, as a huge bonus, detect possible interactions?
*From Pochiraju & Seshadri, Essentials of Business Analytics, 2019, Springer, p 737.
It may pay to make your product less than perfect!
Posted by mark in Consumer behavior, Uncategorized on September 4, 2011
I once analyzed data from a designed experiment that quantified consumer distaste for flaws in chocolate-covered cherries. This was a very rewarding project – lots of free candy! It also produced a counter-intuitive result: People preferred boxes with a few upside-down morsels. I figure this is akin to a beauty mark adding to the enticement of a model or actor. This article on “When Blemishing Leads to Blossoming”, published online by the Journal of Consumer Research confirms that under specific circumstances, a flaw makes a product more attractive. For example, in one experiment (highlighted in the July 16 issue of Wall Street Journal) the researchers (Danit Ein-Gar, Baba Shiv, Zakary L. Tormala) offered either perfect or slightly flawed chocolate bars to several hundred relaxed (strolling around) or stressed (rushing to exams) college students. I searched out the results and reproduced them in this interaction graph from Design-Expert® software. It seems to me that this surprising effect, presuming it’s real, provides yet another devious opportunity for marketing mavens to make us buy stuff. One thing I might advise is that you never buy anything when you are in a hurry.
Management Blog Carnival, Review 3 – Seth Godin blog
Posted by mark in Uncategorized on January 3, 2010
(Note: This blog is the last of three in a carnival organized by John Hunter.
I picked the Seth Godin blog to review because his name rang a bell, but I’d never read anything by him that I specifically recall. As it turns out, I really did not see much that interested me greatly for exactly the opposite reason expressed in this comment by “ariana10” to Godin’s blog of 8/22 (bulleted below):
“I relate to this blog because I am a journalism major at the University of Kansas and I can’t do math for the life of me.”
I am an engineering major who likes math (in moderation!) and I can do it for the life of me. However, I am also keen on marketing and business (MBA, U Minn., ’80) so, even though Godin is light on stats, I must admit that he’s got much to offer for those of us trying to make a living in this high-tech world. Here are a few Seth Godin blogs of 2009 that hit my hot buttons.
- 1/24 Good guys finish… Godin suggests that under the bright light of the internet being generous and fair in business dealings pays off now more than ever. I like that idea a lot.
“When your customer service policies delight rather than enrage, word of mouth more than pays your costs.”
- 8/22 Not so good at math demonstrates the confusion creating by using miles per gallon (mpg) as the metric for fuel efficiency. As I noted in my blog on how the Inverse transformation puts mileage comparisons on track / the best measure for fuel efficiency is gallons per ten-thousand miles.
- 8/28 Spare no expense! does get somewhat quantitative (finally a graph!) in discussing the tradeoffs of giving almost no personal service to a huge number of users (Google) versus a great deal of attention to the troublesome individuals who soak it up.
- 10/26 Dunbar’s Number isn’t just a number, it’s the law is where Godin, a people person, draws the line at 150 – the limit predicted by British anthropologist Robin Dunbar for stable social relationships. I think the number might be a lot less (a tenth?) for engineers than marketers. ; )
What sets Seth Godin’s blog apart from the others I’ve seen (admittedly a small sample) is the amount of original content laced with thought-provoking observations of how people interact and what turns them on or off. He’s a guy worth keeping an eye on, I think.