Local Control Phase Four:  REVEAL
 

Interpret Results!!!

QUESTION: What percentages of patients are predicted to do better on the NEW Treatment than on the CONTROL?  And vice-versa?

ANSWER:  We see in the graph above that these two "better than" percentages add up to only 31%.  The remaining 69% of patients are predicted to have the very same "binary" outcome (survival or not) regardless of which treatment they receive!  And we will see below that it is not clear which 69% of patients those are!  What is clear here is that the overall average difference in six-month-survival (a 3.85% survival advantage for NEW over CONTROL) is not a very meaningful summary statistic.  In fact, any single summary statistic is a huge over-simplification of what the data are actually saying about these two treatments in this patient population.

QUESTION: Are patient X-characteristics consistently predictive of where, within the LTD distribution, a patient is likely to fall?

ANSWER:  The Regression TREE model for predicting LTDs displayed above is slightly better than the simple Covariate Adjustment models we tried, but this model also suffers considerable lack-of-fit (R-squared of only 0.14.)  Thus predictions from models for LTDs are rather weak approximations here.  For example, all five of the final leaf-nodes in the above tree contain at least 61% of patients with an observed LTD of ZERO.

QUESTION: This numerical example illustrates that the LC approach can reveal a salient LTD distribution that is [1] not only rather complex but also [2] not particularly useful in predicting outcomes for individual patients from their observed X-characteristics. In what sense, then, can the overall quality of outcomes research be consistently improved by augmenting traditional statistical analyses with LC strategy and tactics?

ANSWER:  The observed LTD distribution from an LC analysis is fully adjusted for treatment selection bias and confounding as is possible using the observed patient X-characteristics.  Perhaps the most important predictors of outcome remain unmeasured!  On the other hand, failing to find stronger LC evidence regarding "why" patient differential response occurs is better than being naively mislead by estimates and p-values from traditional, global models.

Primarily because of the relatively large size of our example (almost 10K patients), many treatment-related effects could be dubbed "highly significant" using traditional models.  Again, the unadjusted advantage in six-month mortality for treatment over control started out at 2.5% with a p-value < 0.0001.  Suppose a researcher decides to fit logistic regression models of mortality that are factorial-to-degree-two in treatment plus one or more of the 7 observed patient X-characteristics.  This strategy shifts emphasis towards general prediction of mortality and, thus, away from direct (easily understandable) treatment comparisons.  In fact, a researcher may well attempt to summarize findings from such a model as follows: [1] treatment remains significant at < 0.0001, [2] four X-covariates (diabetic, acutemi, ejfract and ves1proc) are also significant (< 0.003) predictors of mortality, and [3] treatment displays a highly significant (< 0.001) interaction with one X-covariate (ejfract.)  In view of our above LC findings that differential response in mortality are minimally predictable, who really cares much about these sorts of model-based findings of limited / oblique relevance?    

In summary, the observed LTD distribution is non-parametric; it is not constructed using any MODEL that predicts outcomes (here, six-month-survival rate or the survival rate difference between treatment groups.)  In fact, the observed LTD distribution is "built up" from observed local-average differences in such a way that the LTD distribution may end up being only a little bit "smoother" than the RAW OUTCOMES DATA that one started with!  Thus, it is typically almost as difficult to accurately model observed LTDs as it is to model any other sort of observed patient outcome.