The randomized trial design can be thought of as a means to answer 2 types of general questions: 1) what is the effect of assigning a treatment?; or 2) what is the effect of receiving a treatment?
In public health, we are normally concerned with the first question—the effect of assigning a treatment. If we implement a prevention or treatment program that is efficacious only under strict research conditions but people in the real world would not receive it for any possible reason, the program will not be effective. This real-world context is termed the “average causal effect” of assigning treatment and is best estimated by the intention-to-treat (ITT) analysis, where participants are analyzed according to the group to which they were assigned.1, 2, 3 It has become the generally recommended approach for randomized trials.4
The ITT analysis estimates the average causal effect of assigning treatment to individuals, regardless of whether or not they adhered to the treatment. This represents a diluted effect when compared with what the effect of the treatment would have been had all participants adhered to their assigned treatment. In other words, the ITT is biased toward the null with respect to the average causal effect of the treatment in the population. Therefore, a different analysis is required if one wants to know the actual average causal effect of receiving a treatment, as opposed to the average causal effect of being assigned a treatment. In this article, we describe contexts when the ITT is not appropriate and why, and identify appropriate alternative analyses.
There are 2 reasons why the average causal effect of receiving a treatment may be more important than the ITT for some people. First, even in the public health domain, investigators may want to know what the average causal effect of a treatment program would be if they could improve participation in the program. For example, an ITT analysis might suggest that a vaccination treatment program was ineffective or minimally effective. However, in reality, the treatment program was very effective in the participants who received it, but the implementation of the program was inadequate. In this context, investigators may want to focus attention on improving adherence rather than on improving the vaccine. Also, the average causal effect of receiving a treatment is of primary interest to a patient deciding whether or not to take the treatment as recommended. This decision should be based on the probability of benefit if they take the treatment, or if they do not take the treatment. Providing a patient with information about the average causal effect of being assigned a treatment (ie, the ITT analysis) may not be very helpful, especially because such an average will vary widely among different implementation contexts even when the treatment effect is constant.
Because the ITT represents a diluted effect compared with the average causal effect of receiving a treatment, it is generally thought to represent a conservative approach, where “conservative” represents an approach that is less likely to accept a novel treatment as effective. For example, if the estimated treatment effect is less than the true treatment effect, the medical community is less likely to accept new effective treatments. In this article, we describe 3 scenarios where the ITT is not a conservative approach, even when missing data due to dropouts or other reasons are properly assessed.2, 3, 5, 6 Although some statisticians may be familiar with these concepts, our objective is to raise awareness among clinicians and clinician-researchers.
First, if the reference (usual care) treatment is more effective than the novel treatment, then the approach is not conservative, even though there is bias toward the null. This scenario is commonly found in “noninferiority” trials (in both strictly controlled efficacy trials or pragmatic trials). Consider a reference group receiving proven usual care treatment by which blood pressure decreases by a mean of 10 mm Hg. In a novel treatment group, the mean blood pressure remains unchanged (ie, the novel treatment is ineffective). With <100% but similar adherence rates, the 2 groups will appear more similar; that is, biased toward the null. Investigators who conclude that the ineffective novel treatment is as effective as the usual care treatment might implement the novel treatment even though it was inferior. Thus, the ITT approach is not conservative in noninferiority trials, as it may result in accepting novel treatments that are less effective than reference/usual care treatments.
Second, bias may be away from the null if the adherence rates differ between study groups while both treatments are equally effective if taken.2 Consider a problem that is universally fatal. When taken, usual care treatment results in survival of 50% of participants. Similarly, when taken, a novel treatment results in survival of 50% of participants. The treatments are thus equally effective. However, consider the context where the proportion of adherence in the novel treatment group is 100% and the proportion of adherence in the usual care treatment group is 50%. An ITT analysis of these data would suggest the novel treatment is twice as effective as the usual care treatment. Although the ITT analysis is correct with respect to the effect of assigning treatment to the entire population on average, individual patients who would adhere to the usual treatment should be aware that the 2 treatments are equally effective if taken as recommended. This is important for clinical care. For example, the usual care treatment might be much less expensive, and therefore, preferable for many patients. More generally, if all treatment options are equally effective, then the treatment associated with lowest adherence for any reason (eg, inconvenience, minor side effects, cost) will appear inferior if clinicians and patients are only provided an analysis based on an ITT approach.
In the above example, both treatments were considered equally effective for simplicity. Now consider a context where the novel treatment is superior to the usual care treatment, such that the novel treatment results in 60% survival and usual care treatment results in 40% survival (risk difference of 20%). If adherence is 100% in the novel treatment group, survival will be 60%. If adherence is 50% in the usual care group, survival will be only 20%. The ITT analysis now suggests the novel treatment will appear even more effective than usual care than it truly is (risk difference of 40% instead of 20%). Thus, the bias of ITT in this context would be away from the null.
In some contexts, the ITT analysis may give a qualitatively different answer and suggest that a treatment is worse than usual care when the novel treatment is actually superior. Consider the same survival rates (60% for novel treatment and 40% for usual care) where adherence rates are now 50% for the novel treatment and 100% for usual care. In this context, the ITT analysis would suggest that novel treatment is inferior (30% survival for the novel treatment group [50% adherence × 60% survival] and 40% for the usual care group) even though the novel treatment is more effective when it is received.
To read this article in its entirety please visit our website.
-Ian Shrier, MD, PhD, Evert Verhagen, PhD, Steven D. Stovitz, MD, MSc
This article originally appeared in the July 2017 issue of The American Journal of Medicine.
