Organized Skepticism
How can the science of tobacco and nicotine serve the public good?
By Clive Bates
There are almost as many definitions of science as there are scientists, but one that I like is “organized skepticism.” This is one of four norms of the scientific ethos proposed by sociologist Robert K. Merton in 1942. Robert May, the former U.K. government chief scientist, explained organized skepticism as “a journey, over time, toward contingent understanding guided by experimental tests and sceptical questioning.” Skepticism underpins the scientific process, and uncertainty is a pervasive, permanent and evolving feature of scientific understanding.
How well are we living up to this ideal? I will break science down into five categories: exploration, observation, interpretation, intervention and values.
Firstly, “exploration.” This is about which research questions are asked—or not asked—in the first place and why. In many ways, this is the most critical part of the scientific process because it can set, shape or sink a policy agenda. It should be driven by organized curiosity, the alter ego of organized skepticism. But too often, the control over the questions is governed by forces other than curiosity in the public interest. Why has there been so little curiosity about the countries with the lowest smoking rates or most rapid declines, such as Sweden? The halving of cigarette sales in Japan in just 10 years has attracted minimal interest (though with some notable exceptions), but why? Are researchers flocking to New Zealand to understand the dramatic decline in Maori smoking driven by the uptake of vaping? No, the research community has been distracted by its favored “endgame” measures.
Another example is the expenditure of at least $100 million researching “very low-nicotine cigarettes,” mainly through a series of trials. The U.S. Food and Drug Administration’s enthusiasm for rulemaking in this area drove the research. But after a company, 22nd Century, put these products on the market, there was minimal interest in understanding the consumer reaction. Why? Surely, this should inform the nicotine rulemaking process. Poor questions can exclude important issues or contain false, unsurfaced assumptions. For example, the U.K. recently asked the research community “What can we do to reduce youth vaping?” But do they want to reduce youth vaping if it increases adult smoking? What about young people who would be smoking if they were not vaping? The question limits the possible answers and excludes much we should be curious about.
A final example is the extraordinary lack of interest in the reasons why there is a demand for nicotine and, therefore, why it is likely to persist. In my view, users experience positive hedonistic, functional and therapeutic effects, which underpin the demand. But curiosity in this area has been suppressed beneath a crude narrative of “addiction.”
Secondly, “observation.” This is the careful business of discovering what is happening through surveys, test measurements, experiments, qualitative studies, etc. Much good work goes on in this area, but it could be far better. Given how much money and public and political concern nicotine attracts, shouldn’t we have better data and faster? The English Smoking Toolkit Survey provides monthly data on nicotine use trends published with only a few weeks of delay between the survey and publication. Why doesn’t every country have a surveillance system like this? Every country has a sizeable share of its population dying from smoking-related diseases; why not have the data to understand the phenomenon? In the energy sector, the International Energy Agency produces in-depth statistics on every aspect of energy use for nearly every country in an advanced Data Explorer. But for nicotine, we have only crude prevalence data and nothing that will show us the evolving trends in nicotine use as it develops and diversifies over the next two decades. The FDA regulates the recreational nicotine market in the United States, yet it has no systematic assessment of how that market is developing. Instead, individual manufacturers must report on their own products. It would be so much better to have one extensive survey, including all the main product types and all those products not authorized but on sale. The FDA’s Center for Tobacco Products receives annual user fees of $712 million yet does not even have a basic and timely picture of the market, including the illicit market, in the products it regulates.
Thirdly, “explanation.” This is where scientists try to determine cause and effect. Did vaping flavors cause teenage vaping? Does vaping cause heart disease? Do more people quit smoking if they have access to vapes, or are vapes a gateway to smoking? Alas, in this domain, skepticism is too often replaced with its inverse, confirmation bias, and its more assertive counterpart, motivated reasoning. There are strong incentives to find or claim causal relationships, even where none exist. Why is it hard to establish causal explanations? I tend to focus on three main issues.
- Confounding. One thing appears to be caused by another, but they are both caused by a third factor. If we observe a 50-year-old vaper for signs of illness, how do we account for the effect of several decades of prior smoking? It is almost impossible to make allowances for that. For conditions that arise from cumulative exposure over decades (like chronic obstructive pulmonary disease or heart disease), it is impossible to isolate the effects of vaping and confounding by prior smoking. Yet, claims are repeatedly made that vaping is a cause of these diseases.
- Reverse causation. One thing appears to be caused by another, but the relationship is the other way around. Some studies show that young people who are exposed to vape advertising are more likely to vape. In these studies, “exposure” usually means recall, the ability to remember seeing an advert. But suppose those inclined to vape for other reasons are more interested in vaping ads and more likely to remember them? The vaping causes the recall of advertising, not the reverse.
- Poor external validity. The experiment doesn’t adequately reflect reality. Some machine measurements of vaping aerosol show relatively high levels of some toxicants. But they have often been run with power and flow rates that greatly overheat the liquids, creating conditions no users could tolerate. The machine registers a toxic exposure, but no human would ever be exposed to it. In a clinical trial, volunteers with a declared intention to quit are paid to participate. They are given free products and incentivized to use them. But how well does that experiment reflect the impact of a new product on people who buy their cigarettes at a corner shop? Hardly at all. Yet a giant edifice of support for de-nicotinization of cigarettes has been built on such trials. Those affected are more likely to switch to illicit or smoke-free products.
Fourth, “intervention.” This is the science of deciding what to do or what happened after a policy was implemented. Curiosity and skepticism seem to be actively discouraged in this area of tobacco control research, though with exceptions. One of the largest tobacco control interventions ever is the 2019 ban on vapes and heated-tobacco products in India, which earned a World Health Organization medal. Yet almost nothing has been done to discover what happened, and the WHO continues to promote these prohibitions. I think this is because, for many in that field, the policy and law are the outcomes, not their actual effects on behavior and, ultimately, health. Too many scientists are regulatory literalists, believing that people will do what the regulators hope they will. Armed with observations of harm and explanations for its cause, they wrongly believe that intervention is simply a matter of removing the cause with a law. Not so. An intervention is a disturbance of a complex system of human, technological and economic interactions. It is vulnerable to at least three forms of unintended consequences: first, adverse behavior change (a vape flavor ban causes more smoking); second, workarounds (a vape flavor ban causes people to mix their own flavors using food ingredients); third, illicit trade (an informal economy develops, possibly with organized crime groups extending their portfolio). Intervention always requires a systems approach and an economic appraisal to assess the likely outcomes and their cost-effectiveness or value for money. Far too many biomedical scientists trespass into the economic discipline of intervention without the necessary skills and experience.
Fifth, “values.” Politicians often declare they are “led by the science.” But no policies can or should ever be determined by science alone. Values or ethics must have a central role—just because something might work doesn’t make it right or acceptable. For example, if we could show that misleading young people about vaping risks reduced youth vaping, would that be an acceptable policy? Should we intervene to stop smoking in private homes? Should children be separated from parents who smoke? Should smokers be denied healthcare for reasons of contributory negligence? There are boundaries to what we find acceptable, whatever the science tells us might work. But different people have different boundaries.
