I've observed a flourishing of private, U.S.-based organizations applying some variant of cost-effectiveness or "value" analysis to medical treatments, and particularly prescription drugs, with some focused specifically on cancer drugs. Peter Neumann and Joshua Cohen noticed this too, and wrote a handy NEJM Perspective that serves as a guide to the programs in this area and their methodologies. (See also the Robert Dubois' article in the Journal of Comparative Effectiveness Research.)
Because "value" is not clearly defined, the five organizations that Neumann and Cohen review take different analytic approaches.
Among the organizations considered by Neumann and Cohen, the Institute for Clinical and Economic Review (ICER) has the broadest purview, conducting evidence reviews and cost analyses of medical treatments. It recently initiated a drug assessment program that will be produce clinical- and cost-effectiveness evaluations, as well as budget impact analysis, of 15-20 drugs within the next two years. ICER's approach considers comparative clinical effectiveness (and the strength of evidence for it); incremental cost-effectiveness; other benefits, disadvantages, ethical, and contextual factors (hereafter, I'll just call these all "additional factors") not captured by clinical- and cost-effectiveness methodology; and short-term budget impact.
The ICER process is not purely formulaic. It includes public input and committee votes on value and strength of evidence. (I am a member of one of ICER's committees.) The additional factors the committee considers vary, as warranted, by treatment and context. They might include, for example, whether treatment outcomes reduce disparities across patient groups, whether treatments facilitate greater productivity through, for instance, more rapid return to work, or whether there are any treatment alternatives, among others. The inclusion of additional factors in the process is important, as a pure cost-effectiveness evaluation can leave important considerations out of the process.
Two aspects of ICER process warrant emphasis because they are approached by other organizations differently, or not at all. First, the effect of additional factors on value is approached qualitatively. They are not included directly into a formula for quantitative combination with other notions of value. Second, an "ICER price" of a drug is not only a cost-effective one, but one that would bring the total cost of its use in line total economic growth (GDP+1%).
The American College of Cardiology and the American Heart Association (ACC–AHA) have proposed to include assessments of value into their clinical guidelines, based on cost/QALY. In contrast to ICER's approach, value assessments do not incorporate additional factors or budget impact.
The American Society of Clinical Oncology (ASCO), and the organizations that follow, focus on cancer drugs. The ASCO framework considers overall and progression-free survival and other clinical benefits, toxicity, palliation, time off all treatment, and cost. Apart from cost, which is reported separately, each of these factors is awarded a score, which collectively sum to 130 points. This is a significant departure from the cost/QALY approach of ICER or ACC-AHA in two respects. First, effectiveness and cost are not combined. Second, points awarded for effectiveness are not in consistent units across categories. As Neumann and Cohen put it,
An analogy would be a scheme to measure a car’s value by adding its safety rating — on a scale of 1 to 10 — to its passenger capacity and gas mileage. A meaningful score should instead account for how much gas mileage buyers would sacrifice to gain an additional seat and how much safety they would sacrifice to increase gas mileage.
Be that as it may, the more consistent QALY-based approaches are not without interpretive and methodological limitations. Additionally, ASCO does not consider additional factors or budget impact.
Evidence Blocks, from the National Comprehensive Cancer Network (NCCN) provides patients with 1-5 scores for each of five cancer drug factors: efficacy, safety, evidence quality, evidence consistency, and affordability. A key difference between NCCN's approach and that of other organizations is that no methodology to combine these scores is imposed or provided. Like ASCO's framework, NCCN's scores have no consistent interpretation across categories and additional factors and budget impact are not part of the assessment.
The Drug Abacus, by Memorial Sloan Kettering Cancer Center (MSKCC) bears some similarity to a few other approaches, but also has unique features. It assigns scores to a cancer drug's efficacy (on survival), toxicity, novelty, research and development costs, target disease incidence, and annual life years lost to the target disease. This "category scores" approach is similar in spirit to that of ASCO's and NCCN's (and therefore, includes some of their limitations), but the similarity ends there. Drug Abacus is an interactive tool that lets the user weigh these categories according to his/her preferences to arrive at a "fair" price.
A somewhat controversial aspect of Drug Abacus is inclusion of novelty and R&D costs as price-modifying factors. Pricing that reflects such factors is "misguided," according to Neumann and Cohen. Their point is that what matters (or should matter) to patients and payers is what a drug does for health independent of what it costs to make. However, a user can give these factors zero weights in Drug Abacus, eliminating their influence on price.
A strength of Drug Abacus is that it includes some additional factors (disease influence and life years lost), though what they are is hard wired, not open ended as ICER's approach permits. Another strength is that it provides a means to integrate across categories to combine clinical, cost, and additional factors, though this weighting can vary by user so it's unclear what "society's" price should be.
Again, budget impact is not included. Like other organizations that do not assess this, Drug Abacus cannot speak to the strain the combination of drug pricing and volume may put on payers. As we've seen in the case of hepatitis C drugs, this is an important consideration, though the practicality of addressing it through drug prices or by, say, finding offsets from less cost-effective treatments within the health system is a worthwhile debate.
My view is that it would be a mistake to rank these five approaches in order of superiority. Each provides a different approach to evaluating drug value, with strength and limitations, as noted. Importantly, none of these organizations have any authority over drug prices or drug coverage. They serve as purveyors of information and analysis. As such, they inform the ongoing debate about the value, price, and budget impact of prescription drugs, and it's up to us—or our elected representatives or the managers of our private insurance plans—whether and how to put the information they provide to use.
Austin B. Frakt, PhD, is a health economist with the Department of Veterans Affairs, an Associate Professor at Boston University’s School of Medicine and School of Public Health, and a Visiting Associate Professor with the Department of Health Policy and Management at the Harvard T.H. Chan School of Public Health. He blogs about health economics and policy at The Incidental Economist and tweets at @afrakt. The views expressed in this post are that of the author and do not necessarily reflect the position of the Department of Veterans Affairs, Boston University, or Harvard University.