The most confusing part of an FDA application explained
By Cheryl K. Olson
In October 2020, the U.S. Food and Drug Administration’s (FDA) Center for Tobacco Products issued draft guidance sharing “current thinking” about “principles for designing and conducting tobacco product perception and intention studies.” As a researcher, I raise a glass in celebration. Unlike toxicology, behavioral science is fuzzy and subject to interpretation. We need signposts to follow.
The guidance applies to premarket tobacco product applications (PMTA), modified-risk tobacco product (MRTP) applications and substantial equivalence reports. If you understand why and how to do tobacco product perception and intention (TPPI) research, your application is more likely to be approved. Let’s stroll through the guidance, highlighting key points and noting ways to put the FDA’s suggestions into action.
Why TPPI?
Behavioral research is one type of evidence the FDA uses to decide if a new tobacco product is “appropriate for the protection of public health.” As the guidance notes, perceptions refers to “beliefs, attitudes, judgments and expectancies” about a tobacco product. In other words, how do you—a tobacco user or nonuser—understand what the label says, what the risks are or who this product is meant for?
Why do we ask about intentions? As the guidance says, “intentions to use tobacco products may help predict future tobacco use behavior,” especially near-future actions. The FDA wants to know whether smokers will switch to your smokeless, electronic nicotine-delivery system (ENDS) or heat-not-burn product (a good thing) and whether nonsmokers or former smokers will take it up (a bad thing). But it’s seldom practical or affordable to follow people around to watch what they do or to survey them repeatedly.
So, we fall back on measuring behavioral intentions as the next best thing. We compare smokers intending to quit—now or within the next six months—to smokers with no plans to quit. We measure whether smokers’ quit intentions change after reading about your product and seeing pictures of your packaging or advertising.
TPPI study aims: your essential roadmap
“[The] FDA recommends that you develop TPPI study aims (the overall goals of the study) prior to conducting the study.” This step is not a nice-to-have; it’s essential. Defining the aims of your study and turning them into answerable research questions creates a roadmap for everything you do. Just as important, study aims tell you what not to pay attention to. If you skip this step, you’ll end up wading through confusing piles of data, perhaps shoveling it all into your application and hoping the FDA can make sense of it.
As you write your survey questions, figure out your sample quotas and review your data tables, you will periodically get lost in the details and need a touchpoint. Referring to your study aims will reorient you and keep you moving in the right direction.
Here’s another way to look at study aims. What product story do you want to tell the FDA? There are some research questions all PMTAs need to address, such as perceptions of risk (more on that later). Others will be specific to your product and your story.
Perhaps your smokeless product appeals more to a particular group, such as female or lower income smokers, than competing products do; your story includes addressing the needs of that underserved group of smokers. Or your high-nicotine vape product is “appropriate for the protection of public health” in part because it attracts heavily nicotine-dependent smokers who, to use FDA-speak, perceive it as an acceptable substitute for cigarettes. If your less-toxic tobacco product appeals to smokers not intending to quit—who are discouraged or unmotivated—that’s a further public health win. If your product and its various flavors do not appeal disproportionately to youth or you have a solid method of restricting youth access, say so.
From market research or conversations with customers, you might have a good idea of your story. But to tell that story to the FDA, you must design research questions and collect data to support it.
Study types (qualitative or quantitative) and methods
Are you telling your story with words or numbers? When you don’t know a lot about perceptions and intentions (say, a new product or target population), start with focus groups or individual interviews. But qualitative doesn’t equal free-form; a structured set of questions, an unbiased interviewer and an analysis plan are a must if the FDA is going to take you seriously.
Some research questions might be answered with experimental studies. For example, you could test effects of new labeling by seeing how subjects randomly assigned to view that labeling answer questions about product perceptions differently than a control group. Another option is an observational study: for example, watching a subject follow directions to operate or charge a new vaping device without prompting from researchers.
Often, your behavioral research aims can be met entirely through cross-sectional quantitative studies: one-time surveys of people who’ve used your product and people who haven’t.
You can get some of the benefits of a qualitative study by adding open-ended questions to your multiple-choice survey. Let’s say you’re surveying users of Vape A. You ask, “Why did you choose to try Vape A? (Select all that apply).” You then invite those who clicked on “To help me quit smoking cigarettes” to “Please tell us more about your experience using Vape A to cut down or quit.” This offers two benefits: It helps you better understand how people use Vape A. And, when paired with solid numbers, user quotes help make your case to FDA reviewers that Vape A deserves a spot on that continuum of risk.
Study measures
Your best bet is to adapt questions from research published in academic journals, especially studies by authors who work for the FDA or other government health agencies. Next best are validated measures from the industry—for example, Philip Morris International has tested questions on addiction and health risk perceptions. Note that your questions need to “be written or adapted in a manner that specifically refers to the product (by name) that is the subject of the study.”
Perception questions should include risks to health and risk of addiction asked in several ways: the absolute risk of using your product; the relative risk as compared to other products in the category as well as cigarettes, nicotine-replacement therapies and no tobacco use; and the effects of dual use (health risks of sticking with smoking, partly substituting your product or switching completely). A good question will be direct, specific and unambiguous: Ask about perceived personal risk of specific types of serious illnesses or earlier death from using Vape A not generically about harm to health.
Behavior intention questions include intent to buy, try, use or stop use of your product. Detailed questions, such as intent to partially or completely replace cigarettes with Vape A, are best left to surveys of Vape A users; it’s too much guesswork for people new to the product. (The guidance doesn’t cover “actual use” studies, but don’t forget to ask your customers about monthly and daily use, reasons for use and use situations.)
Participant sampling and recruitment
Now that you know what to ask … who do you ask, and where do you find them?
For the who part, you need a sampling plan and eligibility criteria. You want to show that Vape A appeals to intended users (smokers or users of other tobacco products) and not to unintended users (nontobacco users and former users, especially youth). Your sampling plan might call for equal numbers of smokers intending to quit, smokers not intending to quit, never-smokers and former smokers—making sure to include a disproportionate number of young adults and a minimum number of people with vaping experience.
Eligibility criteria can be tricky; check the published research. For example, a “never-smoker” is often defined as someone who has smoked fewer than 100 cigarettes in their lifetime!
As to the where … the FDA is open to a variety of modes of data collection (e.g., online, phone, in person) and subject recruitment methods (such as online panels and random-digit dialing). All methods will introduce some bias; online panels may have fewer low-literacy participants, for example. Have a scientific justification for the approach you choose. And phrase your screening questions carefully, so online panel members who just want the dollars can’t guess the “right” answers to join.
Common TPPI mistakes
- Not documenting your research methods (e.g., survey response rates, how you recruited)
- Biased recruiting; for example, if you offer product coupons as an incentive to fill out your survey, people who don’t like your product won’t respond, and your study sample won’t be representative of the larger population. Use neutral incentives (like a choice of retailer gift cards).
- Not enough people in your study—this might be a too-small total sample size or not enough members of important subgroups to detect between-group differences (e.g., women versus men).
- Unclear questions or incomplete response options: Prevent confusion by pre-testing your survey with individuals from your target groups (including less-educated folks).
- Generic questions—the FDA doesn’t want general perceptions of vapes or snus. They need to draw conclusions about your product, including any flavor or strength options.
- Not protecting privacy of research subjects—document your “adequate procedures for human subjects protection;” that is, their “rights, safety and welfare.”
- Treating TPPI studies like market research—the FDA asks that companies “select appropriate study personnel for TPPI studies … who have sufficient formal education, training and experience in conducting social or behavioral science research to ensure the study is designed or conducted appropriately.” Translation: We want to hear from academics like us who speak our lingo.
