Ad Watchers’ episode four of season two provides advertisers insights into presenting evidence to support a claim at NAD. Eric Unis, Senior Attorney at the National Advertising Division (NAD), is joined by colleague Annie Ugurlayan, NAD’s Assistant Director, to walk listeners through what to expect when presenting a claim substantiation case.
Annie begins by explaining that no matter the claim type, the stronger it is, the stronger its supporting evidence must also be. She states that different types of evidence bear different weights. For instance, human trials are more persuasive than animal trials when supporting a claim regarding pharmaceuticals marketed toward people. Annie also points out that it isn’t just express claims that need substantiation. Implied claims hold the same burden of proof. She uses the example of a “nontoxic” claim which would lead consumers to expect no health risks, even transient risks, to occur from the use or misuse of the product. Companies that make such implied claims must be able to provide evidence to support them.
Eric enters the conversation by sharing that industry standards may guide NAD’s expectations for substantiation. However, to do so, the standard must be current and relevant. Practitioners may deviate from the industry standard if their product testing is a better mimic of real-world conditions. He cautions that, regardless of the standard, all studies must be conducted under consumer-relevant conditions. Companies may turn to proprietary testing when an industry standard doesn’t exist, but such testing must be reliable, statistically significant, and consumer-meaningful. Eric then explains that NAD puts more evidential weight on full studies than on abstracts or summaries. The organization prefers to review the methodology to evaluate reliability and relevance.
Eric poses an interesting question to Annie, asking about NAD’s approach to complex scientific issues. Annie explains that, in such cases, the organization allows both sides to bring in expert witnesses to analyze evidence and present an opinion. Expert reports are most useful when no clear scientific standard has been set. However, these reports are not enough to substantiate a claim on their own. Advertisers must also submit true scientific studies. Eric adds that NAD may give an expert opinion little to no weight if it is contrary to scientific consensus. In other words, if an opinion is an outlier among specialists, NAD is less likely to find it reliable.
At this point, the duo introduces distinguished guest Tom Rosholt, the owner and principal of Tom Rosholt Consulting, Inc. His company offers consulting on ad claim substantiation, analytics, and research. After introducing himself, he defines claim substantiation as the task of forging a connection between a message and a metric. His responsibility as a consultant is to ensure that the message and the metric connect before the claim goes to market. If a claim is already in the market, he analyzes data to reveal the strengths and weaknesses of the claim’s supporting evidence.
Earlier in the episode, Eric mentioned the importance of statistical significance, and here, he asks Tom to define it. Tom answers, explaining that it is the scientific equivalent of being able to prove something beyond a reasonable doubt. He then answers Annie’s question, “What is one thing you wish lay people understood about scientific evidence?” by opining that lawyers too often look at statistics as the truth rather than an approximation of the truth. He uses the example of flipping a coin 10 times. The chance of getting heads each time is 50 percent. Though this is the case, that does not mean that the coin flip will result in heads five out of 10 times. In his experience, many fail to consider that the coin could land on heads as many as 10 times or as few as zero.
Tom makes a few recommendations for representatives in claim substantiation cases. Be skeptical and view a claim as unsubstantiated until you see support for it and be aware of bias and error in testing. He ends by explaining that as a statistician, his approach to analyzing data is static and objective. It does not change depending on the product or industry he is researching. His goal is to present the numbers clearly and honestly.
Eric and Annie wrap up the show with the following takeaways:
- Make sure your testing matches the claims and is reliable and consumer relevant. All testing should elicit statistically significant results pertinent to real-world use of the product.
- Expert reports alone do not suffice to provide a reasonable basis. These reports need to provide supporting evidence for any conclusions.
- Bring your statistician into the fold before making a claim. This ensures you have the information necessary to make the right claims and support claims as needed.
- Be skeptical in your approach to claim substantiation. Assume that a claim is not supported until there is evidence to the contrary. Remember that evidence needs to be reliable, and results should be as specific as possible.
Visit to Learn More: NAD FAQs
Contact Information: nad@bbbnp.org
Listen to the full episode here.