Abstract
The debate surrounding whether the findings of efficacy studies are applicable to real-world treatment situations is ongoing. The issue of lack of applicability due to a lack of clinical heterogeneity could be addressed by employing less restrictive inclusion criteria. Given that health economic assessments based on cost-effectiveness measures are required by many governments and insurance providers, the impact of this choice may be far reaching.
The objective of this article was to explore the use of a pilot study to examine the impact of inclusion criteria on cost-effectiveness results and clinical heterogeneity. A health economic assessment was conducted using QRISK®2 and simulation modelling of different population groups within the pilot study in Lower Austria. Patients were referred by their family physicians to ‘Active Prevention’ (Vorsorge Aktiv), a community-based lifestyle intervention focused on exercise and nutritional programmes. Cardiovascular risk factors were recorded before and after the intervention and translated to cardiovascular events.
As expected, enforcing restrictive inclusion criteria produced stronger and more irrefutable computations — in the expected number of events, the number of deaths, the incremental cost per life-year saved and in the 95% confidence interval. These findings provide insight into the issues surrounding clinical heterogeneity and the need for restrictive inclusion criteria. This is not a full health economic assessment of the intervention.
While inclusion criteria provide stronger results by limiting populations to those who would benefit the most, they must be enforced, both within and outside the clinical trial setting. Enforcement has costs, both monetary and arising from unintended negative consequences of enforcement mechanisms. All these considerations will affect the results realized by the payer organization.
A pilot study can reveal whether an intervention may be cost effective ‘enough’ without restrictive inclusion criteria and can enable researchers to search for population subgroups in which the intervention remains cost effective. When the pilot study does not indicate sufficiently strong cost-effectiveness results, the broader trade-offs between clinical heterogeneity and the strength of the submission package to the reimbursement agency can be discussed by all parties. Payer concerns about the ability to generalize the results beyond the clinical trial can also be discussed at this time. Applicability then depends on the ability to enforce inclusion criteria similar to those used in the trials in the real world.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Godwin M, Ruhland L, Casson I, et al. Pragmatic controlled clinical trials in primary care: the struggle between external and internal validity. BMC Med Res Methodol 2003 Dec 22; 3: 28
Rothwell PM. External validity of randomised controlled trials: “to whom do the results of this trial apply?” Lancet 2005; 365(9453): 82–93
Van Spall HG, Toren A, Kiss A, et al. Eligibility criteria of randomized controlled trials published in high-impact general medical journals: a systematic sampling review. JAMA 2007; 297(11): 1233–40
Steg PG, López-Sendón J, Lopez de Sa E, et al. External validity of clinical trials in acute myocardial infarction. Arch Intern Med 2007; 167(1): 68–73
Malmivaara A, Koes BW, Bouter LM, et al. Applicability and clinical relevance of results in randomized controlled trials: the Cochrane review on exercise therapy for low back pain as an example. Spine (Phila Pa 1976) 2006; 31(13): 1405–9
Jones R, Jones RO, McCowan C, et al. The external validity of published randomized controlled trials in primary care. BMC Fam Pract 2009 Jan 19; 10: 5
Gartlehner G, Hansen RA, Nissman D, et al. A simple and valid tool distinguished efficacy from effectiveness studies. J Clin Epidemiol 2006; 59(10): 1040–8
Gross CP, Mallory R, Heiat A, et al. Reporting the recruitment process in clinical trials: who are these patients and how did they get there? Ann Int Med 2002; 137(1): 10–6
Vanderpool HY, Weiss GB. False data and the therapeutic misconception: two urgent problems in research ethics. False data and last hopes: enrolling ineligible patients in clinical trials. Hastings Cent Rep 1987; 17(2): 16–9
Hippisley-Cox J, et al. Predicting cardiovascular risk in England and Wales: prospective derivation and validation of QRISK2. BMJ 2008; 336(7659): 1475–82
Hense HW, Schulte H, Löwel H, et al. Framingham risk function overestimates risk of coronary heart disease in men and women from Germany: results from the MONICA Augsburg and the PROCAM cohorts. Eur Heart J 2003; 24(10): 937–45
The QRISK®2-2010 cardiovascular disease risk calculator [online]. Available from URL: http://www.qrisk.org/ [Accessed 2011 Mar 29]
Berger K, Hessel F, Kreuzer J, et al. Clopidogrel versus aspirin in patients with atherothrombosis: CAPRIE-based calculation of cost-effectiveness for Germany. Curr Med Res Opin 2008; 24(1): 267–74
Levy E, Gabriel S, Dinet J. The comparative medical costs of atherothrombotic disease in European countries. Pharmacoeconomics 2003; 21(9): 651–9
Gandjour A, Lauterbach KW. When is it worth introducing a quality improvement program? A mathematical model. Med Decis Making 2003; 23(6): 518–25
Department fur Evidenzbasierte Medizin und Klinische Epidemiologie. Final report: evaluation of the pilot program Vorsorge Aktiv (’Active Prevention’) [in German; online]. Available from URL: http://www.donau-uni.ac.at/imperia/md/content/department/evidenzbasierte_medizin/projekte/evaluation/vorsorge_aktiv.pdf [Accessed 2011 Mar 29]
Rawlins MD, Culyer AJ. National Institute for Clinical Excellence and its value judgments. BMJ 2004; 329(7459): 224–7
Eichler HG, Kong SX, Gerth WC, et al. Use of cost-effectiveness analysis in health-care resource allocation decision-making: how are cost-effectiveness thresholds expected to emerge? Value Health 2004; 7(5): 518–28
Sculpher M, Gafni A. Recognizing diversity in public preferences: the use of preference sub-groups in cost-effectiveness analysis. Health Econ 2001; 10(4): 317–24
Acknowledgements
Financial support for this study was provided by the
Niederosterreichischen Gesundheits- und Sozialfonds, Stattersdorfer Hauptstrasse 6A, 3100 St. Pölten, Austria. The funding agency is also the sponsor of the pilot study discussed in this article as well as the full-sized clinical trial that ensued. They placed no restrictions or requirements on the publication of this article. The authors have no conflicts of interest.
The results of this study were presented at two meetings: the European Forum for Evidence-Based Prevention (EUFEP) Congress; Baden (Austria); 2009 June 24–26 and the Institute for Operations Research and Management Sciences (INFORMS) Annual Meeting; San Diego (California, USA); 2009 October 11–14.
The authors thank Mark Helfand, MD, MPH, from the Oregon Health and Science University for his thorough review and excellent suggestions for improving the paper.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Rights and permissions
About this article
Cite this article
Richter, A., Thieda, P., Thaler, K. et al. The impact of inclusion criteria in health economic assessments. Appl Health Econ Health Policy 9, 139–148 (2011). https://doi.org/10.2165/11590150-000000000-00000
Published:
Issue Date:
DOI: https://doi.org/10.2165/11590150-000000000-00000