Applying design of experiments to marketing

Design of experiments (DOE) is a term familiar to most quality professionals. Some use it on a regular basis and others try their best to avoid it. Most of those who employ this problem-solving tool have done so mainly on behalf of quality improvement projects. Limiting DOE to just these areas or types of initiatives results in huge lost opportunities for these organizations, purchase not even considering more personal missed opportunities such as the failure to expand the professionals’ own skill sets.

Consider that organizations spend a fair amount of labor and financial resources on advertising, sale specifically as it relates to its timing, generic media usage, channel selection, and so forth. The assumptions that go into the rationale for many of these decisions can be costly and significantly inhibit their effectiveness. Why not apply this experiment-based methodology to understand these relationships? It would illuminate not only their complexity and counter-intuitiveness, but also help determine the effect that advertising, timing, product pricing, and more might have on sales.

Analyzing the value of an increased budget

Joe is the CEO of a large company. The vice president of marketing approached Joe with a request to increase the marketing budget to include additional advertising in twelve marketing areas. As usual, the VP defended his proposal on the basis of a projected increase in sales that he believed it would produce. This is something that came up annually during the budgeting process. However, this year Joe wanted to find out whether the organization received the value proposed by the increased budget—essentially determining if there was an adequate return on investment for the additional expense. This analysis could benefit from a DOE perspective and would be a great opportunity for one of Joe’ Six Sigma Black Belts to apply this tool in a “non-quality” initiative.

Wanting to keep it simple, 50 percent of the requested increase in the advertising budget was approved with the following caveat: Joe, the vice president of marketing, and a Black Belt would collaborate in selecting six of the twelve areas to receive additional advertising and the remaining six areas, with no additional advertising, would be used as the control group. Marketing had already created a forecast of monthly sales in each of the marketing areas, and the intent was to measure the increase in the predicted forecast. Using statistical characteristics of the forecasts, they estimated that they had a 95-percent chance of increasing sales by 4 percent with additional advertising in those six markets.

After additional advertising in the six areas was deployed and six months of data collected, analysis failed to reveal any significant difference between the test and control areas. Although there was a minor increase in sales over the forecast, the increased amount of advertising did not justify the increased amount in sales. Under the past assumptions, without the statistical analysis, the observed increase in sales would have been attributed to the additional advertising.

Going beyond the initial findings

This finding, however, encouraged Joe and his executive team to take a second step and try to determine the proper amount to spend on advertising. They wanted to proceed with caution, however, because they believed that much of their success was due to the quality of their product and the effectiveness of  how they communicated this through their advertising. They decided to conduct an experiment in fifteen market areas—areas where they had been trying to make additional inroads. Their primary goal was to measure the effect of advertising on sales—not on traditional measures such as recall rate or attitude toward their product. This would allow them to also understand the causal (not merely the correlation) connection of advertising on purchases (sales). Correlation could be misleading and become self-serving; historical increases in advertising were based on forecasts of increased sales and not the other way around.

They  agreed on three treatment levels (a 25% decrease, no change, and a 50% increase in advertising budget). Because advertising alone does not determine sales, other important factors considered were point of sales and sales effort. Price was held constant. This led to a designed experiment, with three factors and three levels, conducted over a period of one year. The results did not agree with the ingrained expectations of the organization. In addition, these results did not reveal any interactions between the three factors. However, they did make the organization receptive to the use of DOE outside of “quality.”

Previously, as part of on ongoing marketing effort, the organization mailed out discount coupons of various amounts to be redeemed at specific retail stores; discounts ranged from 2–40 percent. The goal was to achieve at least a 10-percent response rate from the mailings. As the marketing department became more comfortable with the use of quality tools, they began to proactively call upon the Black Belt to help. In this case they wanted to determine the percent discount that should be offered to achieve >10-percent return response rate.

By using historical data and process modeling tools, the Black Belt was able to understand the relationship between discount percentages and customer response. Regression analysis indicated a fairly strong non-linear relationship.

Although a strong correlation was good, the original question still remained unanswered: What discount rate will generate a minimum 10-percent coupon redemption by the customers? Ongoing process modeling activity and analysis was performed to come up with the following results: With 95-percent confidence level, a discount coupon of 18 percent should create at least a 10-percent response from the mailing and could generate as much as a 20-percent redemption. This made the discount coupon process more predictable for budget planning purposes, where ROI could be predicted and measured.

Although these were rather simple applications of quality tools to marketing, it allowed the organization to use the DOE methodology and other quality tools in non-traditional ways: optimizing its marketing mix, setting discount coupon levels, forecasting customer preferences, determining the effect of price on sales, designing marketing brochures to influence conversion rates, etc.

Bottom line: Leverage the skills of Six Sigma Black Belts (internal and/or external) for more than solving “quality” problems.