The status of health economic evaluation within decision making in Austria

Economic evaluation is defined as “the comparative analysis of alternative courses of action in terms of both their costs and consequences” [6, p. 4]. Thus, it establishes the relative costs and impacts of health interventions, with the underlying objective of maximizing population health for the available resources [7]. This includes identifying, measuring, and valuing costs and outcomes of the alternative interventions that are considered within such an analysis. Different types of economic evaluation exist, whereby CEA and CUA (which is sometimes classified as a sub-group of CEA) are used most widely. Both of them measure costs in monetary units, but they differ in the way they measure outcomes. While the former quantifies outcomes in the form of “natural units” (e. g., life years gained, depression-free days reduced), the latter determines the benefit of an intervention in the form of utilities, whereby the term “utility” refers to the “preference individuals or society may have for a particular set of health outcomes” [6, p. 14]. The attraction of utilities has been that they allow inclusion of health-related quality of life issues and at the same time introduce a generic outcome measure. One of the most often used utility measures in economic evaluations are “quality-adjusted life years” (QALYs). In QALYs, length of life and health-related quality of life are consolidated into a single value [8].

The results of a CEA or CUA are described in terms of the ratio of incremental costs per unit of incremental health benefit, the incremental cost-effectiveness ratio (ICER). The results can be classified within a so-called cost-effectiveness plane (Fig. 1). If an intervention is less costly and more effective (south-east quadrant) or, vice versa, if it is more costly and less effective (north-west quadrant), the interpretation of the result is straightforward. The former option suggests adopting the intervention, while in the latter case, it may be rejected. However, most often, the intervention is both, i.e., more effective but also more expensive than the alternatives (north-east quadrant). In this case, a decision rule is required. If the aim is to produce the maximum health within a population, then the criterion for adopting the intervention will be that the intervention evaluated is a better use of scarce resources than spending the money on something else (opportunity costs). To address this, the cost-effectiveness threshold has been developed which is “an estimate of health forgone as other activities are displaced to accommodate the additional costs of new technologies” [9, p. 1]. Therefore, adopting technologies with an ICER below the cost-effectiveness threshold means that the health gains will outweigh the decrements, while it is the other way round if a technology with an ICER above the threshold is reimbursed. This shows that the aim of economic evaluations is not to save money but to gain as much health as possible from the available resources [7]. The underlying utilitarian principle of maximizing health means that CEA and CUA fall within the field of normative economics (as opposed to positive economics) [10]. Thus, they recommend specific policy actions based on the normative judgement that one goal of health policy ought to be maximizing population health [11].

Economic evaluations have been widely applied for informing reimbursement decisions in publicly funded health care systems in Europe, but also in countries outside Europe such as Australia or Canada [3, 12]. The most commonly used types are CEA and CUA [13]. However, the extent to which they are used varies. For example, in the Netherlands (in inpatient care), Sweden, or the United Kingdom, its use is extensive, while in Germany or the United States, it is rather limited [5, 14]. The field in which economic evaluations have been used most extensively is drug reimbursement decisions. The first country that included economic evaluations as a requirement in the submission guidelines for the pharmaceutical industry was Australia. According to Drummond (2013) [12], half of the countries in the European Union request economic analyses to varying degrees (e. g., for high-cost medicines in the Netherlands or France, for drugs with substantial additional clinical benefit in Austria and France [5]). More recently, countries in Latin America and Asia have also expressed interest in using economic evidence more systematically in coverage processes [12].

Considerable differences exist regarding the way such studies are applied in the reimbursement processes. Some jurisdictions (e. g., Italy, Spain)—while overall committing themselves to consider efficiency in their decisions—have not defined formal requirements for the manufacturers on how to conduct economic evaluations. Others (e. g., Scotland, Canada, Belgium) have defined more precise methodological guidelines and procedural arrangements; however, there are still differences concerning the range of technologies (e. g., all drugs or only a selection) that are subject to economic evaluations [12]. Variations also exist regarding the appraisal of the submitted manufacturers’ studies. In some cases this is done by the staff within the government or an HTA agency (Finland, Norway, Sweden), in others by a separate group (e. g., from academia) [5]. Appraisal of the economic evaluations in some countries means conducting an own study and comparing it with the submission (e. g., in England), while in other jurisdictions (e. g., Australia, Canada, the Netherlands) it refers to critically appraising the submitted studies [5]. Finally, differences exist concerning the extent to which the committees that are involved in the decision processes represent expertise in economics, which has implications for the interpretation of the studies and the weight they may have in the decision making itself [12].

Little evidence exists regarding the impact of using economic evaluations in decision-making. It was stated that whether allocation of health care resources has changed or whether health overall has improved as a result of using economic evaluations is difficult to answer, because the counterfactual data (how would decision-making have developed if economic evaluations had not been incorporated) is unknown [12] and too many different factors influence the final decisions [15]. However, economic evaluations require specific types of data which may have contributed to improvements in clinical study design (e. g., more focus on patient-relevant outcomes in clinical trials, more head-to-head trials) and to the development of methods (e. g., network meta-analysis in the absence of head-to-head trials) [12]. Furthermore, economic evaluations may have contributed to targeting interventions to specific sub-populations (identified in economic evaluations) and to an increasing use of managed entry agreements [12]. It was demonstrated that in countries that use economic evaluations, the ICER is a prominent factor in decision making; however, decision makers consider other criteria (clinical need, degree of priority etc.) as well [12, 15]. Applying a transparent decision criterion such as the ICER has resulted in high-cost interventions being more evenly distributed across the country in contrast to “postcode rationing” [12].

A large number of challenges for using economic evaluations in decision making have been described in the literature. According to their nature, they can be classified into different categories (see Table 1) that are described below.

Austria has a social insurance-based health care system. Therefore, it can be classified as a Bismarck system according to the typology outlined above. However, in addition to health insurance—which funds 44% of health care expenditure—taxes and private sources play an important role, with a share of 30 and 26%, respectively. The overall health care spending in 2016 was 36.9 billion, representing 11.2% of the gross domestic product. Furthermore, the system is considerably fragmented in terms of governance and service provision. For example, the health insurance is responsible for outpatient services (including medication), while responsibility for hospitals (both hospital inpatient and outpatient settings) is mainly at the regional governance level. Responsibilities of preventive activities differ according to prevention type. Some rest within the federal government (e. g., vaccines), others within the health insurance (e. g., certain screening programs), but funding may be split between all of them (Fig. 2).

The role of health economic evaluations in pricing and reimbursement decisions differs across decision processes. Fig. 2 presents an overview on the different processes for coverage decisions. It shows that separate processes exist for pharmaceuticals and non-pharmaceuticals, and even within pharmaceuticals, processes are different between drugs paid by the health insurance for the outpatient sector and drugs used in hospitals. In none of the processes described do formal requirements to use economic evaluations exist, except for the decision-making process for outpatient pharmaceuticals, which falls under the responsibility of the Main Association of Austrian Social Security Institutions. In this process, the rules of procedure specify that each pharmaceutical product for which reimbursement is claimed needs to undergo economic evaluation. However, the term “economic evaluation” in this case primarily means executing price control based on the added clinical benefit of the new drug compared to existing pharmaceutical alternatives [25]. Economic evaluations as defined in this paper (CEA or CUA) are labelled “pharmacoeconomic studies” in the legal rules of procedure. They are required in two types of submissions: a) if the manufacturer claims that the drug offers a substantial additional benefit compared to existing drug treatment alternatives for all or a subgroup of patients for whom the drug is licensed for; and b) if the drug is submitted for listing in the “yellow box”¹ of the code of reimbursement (Erstattungskodex, EKO²) and there are no alternative drug treatment options already listed [25]. The studies need to be submitted by the manufacturer as part of the dossier.

In the submission manual, basic reporting requirements for economic evaluations are outlined (title; research question; perspective; alternatives analyzed; type of economic evaluation; source of data used; quantitative results of patient benefits and costs; quantification of costs disaggregated by type of cost, quantities, and prices as well as direct and indirect costs; discounting; sensitivity analysis; summary of results) [25]. However, the manual does not specify methodological requirements, except for the types of costs that need to be taken into account (direct costs related to services paid by the health insurance, hospital costs, medical rehabilitation) and costs that are to be excluded (out-of-pocket payments). While methodological issues have partly been addressed in a consensus document initiated by a private industry consulting institute [26], no formal guideline specifying methodological details exists to date. Compared to many international guidelines, the document does not provide precise methodological guidance and leaves a lot of room for flexibility (e. g., concerning the outcome parameter used, the methods of sensitivity analysis applied, etc.) [27]. Its use is not mandatory and rather meant as a recommendation.

Little is known about the exact use of pharmacoeconomic studies in the internal processes and the role they play for the recommendations made by the drug evaluation committee. According to a recent report on drug reimbursement in Austria, no methodological handbook describing the methods that are applied for critically assessing the pharmacoeconomic studies and for summarizing the study results for the drug evaluation committee exists so far [5]. Both the reports for the drug evaluation committee as well as the recommendations the committee makes are confidential. The management of the health insurance makes the final decision on the inclusion into the EKO [25]. Few data are available on the role the pharmacoeconomic studies play in legitimating the final decision and/or in defining subgroups for which the drug may be publicly funded. A study from 2006 demonstrated that in almost all cases where pharmacoeconomic studies were part of the dossier, their use in decision making was rather restricted because of limited relevance and credibility of the study [28]. The degree of usage may have changed since then; however, systematic evaluations on the current use of economic evaluations in outpatient drug reimbursement processes are lacking.

In all other coverage processes described in Fig. 2, economic evaluations are used on a case-by-case basis only, if used at all. They have so far not played a role in defining the non-drug services covered by the health insurance in the outpatient sector. The situation is similar for coverage of services within the hospital, where different processes exist for hospital inpatient (separate for devices and drugs) and for hospital outpatient services. While benefit assessment of devices has gained considerable importance and has been increasingly standardized [29‐31], cost-effectiveness evidence is not used in any steps of the process in a systematic way. One area where economic evaluations have played some role are coverage decisions for vaccination and for screening programs, although again, no standardized processes are in place. Three case studies demonstrate of how economic evaluations have been used in different coverage processes.

When contrasting the Austrian system and the three case studies with the general challenges for using economic evaluation in the previous section (“Challenges to the application of economic evaluations”), the biggest limitation seems to be inherent in the Austrian health care system characteristics. Firstly, the Bismarck system means that—although health care resources are of course not infinite in Austria—the available budget is not as fixed in advance as it is the case in a Beveridge country such as the UK. However, as other cases with Bismarck-based systems demonstrate (Belgium, the Netherlands), a health insurance-based funding in itself does not seem to be the key challenge. It seems that rather the combination with other system characteristics that are inherent in the Austrian system such as fragmentation of funding, governance, and service provision play an important role. Not least, lack of transparency in the decision-making culture has been described as being a core barrier to the inclusion of (cost)-effectiveness evidence into the decision-making processes [24, 36]. For example, resource allocation decisions are often passed on to the level of the clinicians rather than addressing them at the macro-political level (especially in the inpatient sector [37]), which prevents the use of economic evaluation. Several initiatives have now started at the hospital-provider level to systematically evaluate high-cost hospital drugs before funding approval; however, CEA and CUA have so far not systematically integrated into the evaluation concepts [38]. In the case of outpatient drugs, where a form of positive list exists, according to the Austrian law, drugs that are excluded from the list can still be publicly reimbursed if approved by the chief medical officer (based on the individual needs of a patient). Thus, both the culture and the legal basis that form the Austrian health care system hinder a rational approach to resource allocation and thereby the use of economic evaluation. This does not mean that rationing is absent, but it rather indicates that rationing, which de facto always exists in a situation where demand exceeds supply, occurs implicitly.

The case studies demonstrate that efficiency generally has a low priority compared to other reimbursement criteria. As shown in the case of HPV vaccination, the ICER did not play a role in the discussions, while the burden of disease (cervical cancer incidence and mortality in Austria), the long-term impact on cancer epidemiology, safety issues, and the budget impact were all subjects of the political discourse [39, 40]. Even in the absence of a threshold, cost-effectiveness studies could be used to identify preferable patient subgroups. The final decision to publicly pay for vaccinating boys and girls despite a very high ICER demonstrates that this type of evidence is of low priority in justifying decisions. The case rather indicates that the health benefit, affordability, and prices are the core criteria for decision makers, rather than cost-effectiveness. This is also confirmed by the TAVI case, as it has been shown that decisions are based rather on clinical parameters alone, such as severity of aortic stenosis, age, surgical risk, life expectancy, and comorbidities, than on cost-effectiveness results. Efficiency seems at best to be used retrospectively to justify resource allocation decisions that were originally made based on affordability grounds or based on limited capacities that were required to restrict access to a manageable number of patients.

Other challenges outlined in “Challenges to the application of economic evaluations” also seem to be confirmed. Regarding applicability, data quality, and reporting/communication, members of the drug evaluation committee pointed out early that reasons for rejecting the use of economic evaluations were that the presentation of results was unsatisfying and difficult to understand. Furthermore, studies lacked transparency and the application of them to the decision problem was very limited because an inappropriate comparator was used or the study was not based on relevant Austrian data [28]. Some of these deficiencies still seem to exist. As the diabetes case study demonstrates, one reason for neglecting the study may have been its lack of transparency. Furthermore, the outcome parameter (€ per % of HbA1c reduction) and the time horizon (1 year despite the fact that diabetes is a chronic disease) used may likely have been meaningless for the decision problem, all of which resulted in low relevance for the Austrian decision context and low credibility of the study in general. Mayer et al. [17] showed that there are particular quality problems with cost data in Austria because of inconsistencies in the costing methods and cost-reporting standards. This poses an additional limitation to using reviews of economic evaluations (such as the TAVI case presented earlier), because it restricts the generally limited transferability of international study results even further.

Furthermore, the lack of methodological expertise and knowledge seems to be relevant in Austria. Members of the drug evaluation committee expressed concerns regarding their expertise in economic evaluations in a survey in 2006 [28]. Although more than 12 years have passed since this survey, none of the members of the drug evaluation committee is a (health) economist by training. The lack of knowledge is also visible in other decision processes, as an interview among members of the Austrian vaccination committee demonstrated [39]. The interviewee claimed that the HPV study demonstrated that the vaccination is not cost effective. However, this was mentioned nowhere in the report. In fact, in the absence of a cost-effectiveness threshold, a judgement on cost effectiveness based on the ICER is not even possible and has made interpretation of studies difficult (also demonstrated in the TAVI case). It seems to be common that economic terms are often used without a clear understanding of their meaning, which confirms the international findings outlined in “Challenges to the application of economic evaluations.”

The analysis has demonstrated that the challenges with using economic evaluations that have been identified in the international literature seem to also exist in Austria. It is therefore not surprising that economic evaluations have played a negligible role in Austrian coverage processes so far. Some of the challenges could be overcome quite easily if there is agreement among decision makers to systematically use economic evaluations as a source of evidence in reimbursement processes. For instance, education and capacity building will reduce the lack of knowledge among decision makers and increase the capacities of skilled staff involved in critical appraisal of industry studies, which, in turn, will reduce wrong perceptions and misunderstandings. Furthermore, standardization of methods and presentation of results as well as detailed quality criteria will increase the methodological quality of studies, their relevance for the decision problems, and the trust in the studies. To increase transparency, manufacturers could be obliged to submit not just the final study, but also the calculations the study is based on, as is the case in other countries (e. g., UK, the Netherlands, Finland, Sweden), where the decision analytic model has to be submitted alongside the study report. The methodological quality could be increased by a mandatory detailed guideline that clearly sets out the methodological standards and the reporting requirements for industry submissions, but also for economic studies that are commissioned to independent bodies. This shift towards more transparency of the studies would also reduce the perceived risks that economic evaluations may be misused. The submitted models may then be used by the decision makers themselves for analyzing different scenarios (e. g., different price scenarios).

However, the biggest barrier to integrating economic evaluations in reimbursement processes seems to lie within the nature of the health care system. On the one hand, the nature of the studies and the theoretical and ethical principles they are based on are in part incompatible with the Austrian legal context, the system characteristics, the moral values entrenched in the system, and the political culture. As a consequence, some of the currently existing methodological requirements for economic evaluation in Austria (e. g., to apply a health care system perspective and ignore out-of-pocket costs [25]) are in fact debatable within a Bismarck type of health care system. On the other hand, further neglecting of the notion of efficiency will threaten long-term sustainability of the health care system.

Copying how economic evaluations have been integrated into decision making elsewhere will likely fail, even if they refer to the same health care system type such as the Netherlands or Belgium, which are also rooted in the Bismarckian tradition [23, 41]. The situation rather calls for a process that takes the system characteristics into account. Increasing the use of economic evidence first of all requires an understanding among decision makers at all levels that addressing efficiency as one of a number of other decision criteria (and the rationalization and rationing issues that this brings with it) is not unethical, but that it may be rather unethical to ignore efficiency [23]. By passing on resource allocation decisions to clinicians, it remains nontransparent what type of criteria are applied and whether they reflect societal values (e. g., postcode rationing, allocation of resources based on health literacy, etc.). The value of CEA or CUA for decision makers in that sense is that it contributes to accountability by reassuring payers that their money is being spent wisely and by reassuring patients, caregivers, and the general public that their contributions to the health service are treated consistently [3]. The premise is, however, awareness that by using CEA or CUA, some discourse about willingness-to-pay based on a utilitarian philosophy would have to be introduced and that maximization of health would have to become one explicit out of several further decision-making criteria.

To tackle the applicability issue, a process needs to be initiated aiming at identifying what the objectives and needs of the decision makers precisely are [18, 22], i. e., what type of economic evidence they would find helpful and in what way this should be presented to facilitate its use. Both decision makers from various parts of the system and experts in economic evaluation and other forms of efficiency analysis would need to take part in this process. Existing reviews from other countries’ guidelines [42] and suggestions that have already been developed for Austria based on international examples [43] can serve as a starting point. But as the German case has shown [44], the process will also need to facilitate methodological discussions and it will need to be open towards adaptive and new forms of economic evaluations that may differ from the methods that are currently applied in other jurisdictions.

A starting point may be to draw on the suggestions from Williams and Bryan (2007) [11] that are to shift away from normative to more positive economics. Rather than a CEA or CUA, the primary type of economic evaluation to be used would then be a cost-consequence analysis that displays different effects and costs in a disaggregated way without combining results into a single indicator (such as an ICER). Instead of defining cost effectiveness based on willingness-to-pay thresholds in a normative (prescriptive) way, such studies describe the likely consequences in terms of costs and health which may be more approachable by decision makers [45, 46]. However, this needs to be weighed against the disadvantages of cost-consequence analysis: disaggregation results in limited comparability and generalizability; additionally, the absence of a single cost-effectiveness ratio may lead to less consistent decision making. Finally, leaving the weighing of the relative importance of different costs and benefits to decision makers bears the risk of cherry picking [45, 46]. Nevertheless, presenting costs and a range of outcomes in disaggregated form increases transparency, mitigates uncertainty, or makes more clearly visible what the core uncertainties are. Thereby, decision makers are supported with additional knowledge to be used to make informed decisions and to justify the decisions later on, including the value judgement they are based on. As the HPV case study has shown, the disaggregated presentation of knowledge may serve other purposes than just supporting efficient resource allocation (e. g., providing information on equity and affordability, requirements for program implementation, long-term health gains to be expected, etc.). Hence, it reduces the asymmetry of information that usually exists between payers and providers of interventions [47].

Regarding the process of conducting individual studies, the studies will likely be more relevant if decision makers and those who conduct the studies engage more actively in a communication process before the research starts, whereby the parameters to be addressed in the economic study are clarified (e. g., the type of comparator or the outcome measure used).

In parallel to discussions on efficiency and economic evaluation, a discussion on further attributes of benefit that may be valued alongside health gains (e. g., severity of a disease) needs to be initiated and criteria to represent those attributes need to be defined and made transparent.