Introduction

Prescribing medication is an error prone process and errors frequently result in patient harm. Of all acute hospital admissions 5.6% is medication related, and almost half of them is potentially preventable [1]. In the Netherlands, electronic physician order entry with medication surveillance is obligatory and does prevent medication errors considerably [2]. However, medication surveillance is limited to drug-drug, drug-disease, dosing, duplicate therapy, pregnancy, lactation and allergy alerts. Many other prescription errors, not caught by medication surveillance, are possible.

Clinical Decision Support Systems are designed to alert physicians and pharmacists based on algorithms in which patient characteristics are taken into account and may prevent errors not caught by medication surveillance. These algorithms identify patients exposed to potentially preventable risks based on patient characteristics, the absence of co-prescribed drugs and laboratory values. Many Clinical Decision Support Systems do improve practitioner performance [3].

The way physicians and pharmacists are warned does differ substantially between the Clinical Decision Support Systems. Kawamoto et al. reviewed 70 studies on Clinical Decision Support Systems and evaluated the features critical to success [4]. They identified that alerts should be generated automatically as part of clinician workflow, at the time of decision making and provide actionable recommendations. However, presenting alerts to practitioners too often may result in important alerts being ignored along with unimportant ones, so called alert fatigue [5].

Clinical Decision Support Systems that are designed to prevent prescribing errors often generate lists of patients who fulfill the criteria of the algorithm at certain time points during the day. These patient lists are reviewed by a pharmacist whether the patients are actually at an increased risk of potentially preventable patient harm. If needed, the pharmacist contacts the physician by telephone with a recommendation, for example to change pharmacotherapy or to do laboratory assessments. This is a rather time consuming process and the physician is disturbed while performing other activities. Ideally, these recommendations are given to the physician while entering medication orders. At that time, he or she has recently made decisions on pharmacotherapy, is fully aware of the patient’s medical condition, and is working in the patient’s medical record. In our hospital, we implemented a clinical decision support system, that alerts the physician directly after ordering medication. A pop-up is shown, if the patient fulfills the criteria of the clinical decision support system’s algorithm. The pop-ups are shown just after signing the medication order or orders. They are integrated in the hospital information system, and physicians are familiar with these types of pop-ups. There are several importance levels for these pop-up, varying from green pop-ups for informative alerts to red pop-ups for high-level importance pop-ups. The pop-ups of our clinical decision support system were in the medium level and colored yellow. The pop-ups for medication related interventions contain a short description of the possible adverse effect with a clear recommendation and, if applicable, the opportunity to order medication that is missing. By reducing the time needed to manage these pop-ups, we expect to improve the compliance.

In our hospital, we evaluated three algorithms that were developed by the Dutch Association of Hospital Pharmacists. These algorithms recommend to co-prescribe folic or folinic acid with methotrexate, to co-prescribe vitamin D with bisphosphonate, and alert if sodium lowering drugs are prescribed in patients with hyponatremia. We evaluated the change in compliance with these recommendations, after implementation of the clinical decision support systems that show pop-ups to the physician directly during the medication ordering process.

Methods

This study was initiated by the Pharmacy Foundation of Haarlem Hospitals, the hospital pharmacy servicing the Spaarne Gasthuis in Haarlem/Hoofddorp, the Netherlands. The Spaarne Gasthuis is a teaching hospital with 1037 beds. The hospital information system Epic is used, which is an integrated computerized physician order entry system and includes functionalities for the implementation of clinical decision support.

We evaluated three algorithms that are suitable for implementation in a clinical decision support system alerting the physician during the prescribing process. The first algorithm shows a pop-up alert if oral or subcutaneous methotrexate is prescribed without co-prescription of folic or folinic acid (Fig. 1a). These administration routes are chosen, to exclude intravenous methotrexate administrations for oncologic indications. Folic acid or folinic acid should be prescribed to avoid the adverse effects by methotrexate [6]. The second algorithm shows a pop-up alert if a bisphosphonate is prescribed, without co-prescription of vitamin D or analogue (colecalciferol, alfacalcidol, calcitriol or dihydrotachysterol) (Fig. 1b). Addition of vitamin D is advised in the Dutch guideline for osteoporosis and fracture prevention [7]. The third algorithm shows a pop-up alert if one sodium lowering drug is prescribed while the patient has a sodium level of 130 mmol/l or less, or if two or more sodium lowering drugs are prescribed, while the patient has a sodium level of 135 mmol/l or less (Fig. 1c). As sodium lowering drugs, the diuretics, NSAIDs, SSRIs, venlafaxine, carbamazepine, oxcarbazepine, cisplatin, carboplatin were included. Simultaneously with the introduction of the algorithm for hyponatremia, all drug-drug interaction alerts for combinations of drugs that lower sodium levels were filtered and not shown to the practitioners.

Fig. 1
figure 1

a Flow diagram for the clinical decision support system for co-prescription of folic or folinic acid with methotrexate. b Flow diagram for the clinical decision support system for co-prescription of vitamin D with bisphosphonates. c Flow diagram for the clinical decision support system hyponatremia

Full size image

In the pop-up alerts for methotrexate, it is possible to order a prescription for folic acid or folinic acid with two clicks from the pop-up (Fig. 2), and similarly it is possible to order vitamin D from the pop-up alert for bisphosphonates. In the pop-up alert for hyponatremia, the most recent laboratory result is shown. If the physician decides to neglect the alert, a reason should be given with the buttons in the alert. These reasons are similar to the reasons that should be given to override a medication surveillance alert. All alerts that were overridden by the physician, were reviewed on a daily basis by a pharmacist. If relevant, the physician was contacted by telephone.

Fig. 2
figure 2

Alert for the clinical decision support system shown to the prescriber

Full size image

We performed a prospective intervention study with a historical control group. The clinical decision support system was implemented on August 25th 2015. We analyzed all alerts shown in the period between August 25th 2015 until October 11th 2015 (48 days). We compared these results with the time period from July 13th until August 24th(43 days), when the hospital information system generated alerts, but did not show them to the prescribers. These alerts were available for analyses. We analyzed the compliance with the recommendations. For methotrexate, we analyzed whether folic acid or folinic acid was co-prescribed within 48 h after the alert. For the bisphosphonates, we analyzed whether vitamin D or analogue was co-prescribed within 48 h after the alert. For hyponatremia, we analyzed whether one or more orders for the sodium lowering drug were stopped within one hour after the alert.

Data were retrieved from the reporting section in Epic and analyzed using Microsoft Excel version 2007. The change in co-prescription of folic or folinic acid, vitamin D and the number of stopped orders for sodium lowering drugs before and after implementation was analyzed using the Pearson’s Chi-square test, using SPSS version 18.0.

Implementation of the clinical decision support system was part of improving usual care, and therefore no approval of an ethical committee was needed.

Results

We analyzed a total of 1031 orders for methotrexate, 1070 orders for bisphosphonates and 13,412 orders for sodium lowering drugs in the time period before and after implementation (Table 1). For methotrexate 34 alerts were shown (3.2%), for bisphosphonates 107 alerts (10.0%) and for hyponatremia 228 alerts (1.7%).

Table 1 Baseline characteristics
Full size table

In the study period of 43 days before implementation, 179 alerts were generated but not shown to the physicians (Fig. 3). Although no alert was shown, folic acid or folinic acid was co-prescribed after a methotrexate order within 48 h in 7 of the 13 cases (54%). After implementation, folic acid or folinic acid was co-prescribed within 48 h in 19 of the 21 cases (91%). This increase was statistically significant (p = 0.014). For bisphosphonates, the implementation of the alerts resulted in a statistically significant increase in the co-prescription of vitamin D or analogues from 5 of the 47 cases (11%) before implementation to 24 of the 60 cases (40%) after implementation (p = 0.001). For hyponatremia, before implementation 119 alerts were shown and 3 times (3%) an order for a sodium lowering drug was stopped within one hour, while after implementation 109 alerts were shown and 15 times (14%) an order was stopped (p = 0.002). After implementation 106 drug-drug interaction alerts were suppressed.

Fig. 3
figure 3

The change in compliance with the clinical decision support system recommendations

Full size image

After implementation of the alerts, all overridden alerts for inpatients were reviewed by a pharmacist. For methotrexate no phone calls were made to physicians. After review of the overridden bisphosphonates alerts, physicians were contacted six times by phone with the advice to prescribe vitamin D, and this resulted in a co-prescription of vitamin D once (16.7%). For hyponatremia, two times a physician was contacted by phone with the advice to stop the sodium lowering drug, but these advices were ignored.

Discussion

Our results suggest that a clinical decision support system that shows a pop-up during the prescribing process is an effective way to improve prescribing behavior. For all three algorithms tested, the compliance increased significantly. The degree of compliance differed substantially between these algorithms. The co-prescriptions for folic or folinic acid with methotrexate increased from 54 to 91% after implementation. We identified only two cases in which folic or folinic acid was not co-prescribed after implementation. One patient was treated for lymphomatoid papulosis, for which co-prescription of folic or folinic acid is not recommended according to the guidelines. The other case was a child who received on oral instead of intravenous dose of methotrexate for the treatment of acute lymphoblastic leukemia once, and folinic acid was co-prescribed according to protocol. In our opinion the compliance with this algorithm was high, because for most indications co-prescription of folic or folinic acid with methotrexate is mandatory. The co-prescription of vitamin D with bisphosphonates increased from 11 to 40%. Various reasons could justify the ignorance of this recommendation. For example, if vitamin D plasma levels are within the normal range, there is no need to prescribe vitamin D. Another reason is that patients are admitted or seen for other indications than the treatment of osteoporosis, and the physician might not want to interfere with medications prescribed by another physician.

The alert for sodium lowering drugs in patients with hyponatremia increased the number of stopped orders from 3 to 14%. This means that the majority of sodium lowering drugs were continued. There are many causes for hyponatremia other than the use of sodium lowering drugs, such as polydipsia and liver failure. If there is an apparent cause for the hyponatremia which is treated, there might be no reason to discontinue the sodium lowering drug. If the drug is the cause of the hyponatremia, a balance between risks and benefits should be made, and continuing the drug will be justifiable in many cases. For example, furosemide is a sodium lowering drug and discontinuing this drug in patients with decompensatio cordis may result in dyspnea that is much more life-threatening than hyponatremia. Nevertheless, the pop-up resulted in reconsideration of the prescribed drugs in a substantial number of cases, and discontinuation in 11 % of the prescriptions.

In a previous study, four different alert presentation methods for preventing prescribing errors were compared; pharmacy intervention, physician alert list, electronic health record section and pop-up alerts [8]. The pop-up alerts were the method with the highest compliance. Forty-one percent of the recommendations were accepted, compared to 19 to 33% for the other methods. The increase in compliance was 22% for the pop-up alerts. However, in their study the pop-ups were shown while the physician was evaluating the patient record and not in the workflow of the medication ordering process.

In our study, the pop-ups shown during the prescribing process improved the quality of prescribing. The compliance rates were increased by 47, 29 and 11%. Moreover, for folic or folinic acid co-prescribed with methotrexate, almost all advices were accepted, except for two cases in which there was a clear reason for not prescribing methotrexate. Horsky et al. describe designs in the clinical decision support systems that attribute to usability [9]. Various design attributes suggested by Horsky et al. were present in our clinical decision support systems and may contribute to the increase in compliance rates. First, a short general description of the possible adverse effect was present in the pop-up alerts in our study. Second, in the pop-up alert for sodium lowering drugs the last measured sodium level was given in the pop-up, so the physician was aware of the severity of the hyponatremia while prescribing a sodium lowering drug. Third, in the pop-up alert for methotrexate and bisphosphonates, physicians could order folic or folinic acid or vitamin D directly from the pop-up. This assures that the physician does not forget to order these drugs after finishing the initial order and that the correct drug is ordered.

For inpatients, all overridden alerts were reviewed by a pharmacist on a daily basis. As a result, eight times a physician was contacted by telephone, but the recommendations were followed only once. These numbers also indicate that the alert was ignored with a good reason. The rate of sodium lowering drugs stopped due to hyponatremia increased with 11%. However, this alert substituted the drug-drug interaction alerts between sodium lowering drugs, and the number of alerts shown to physicians was similar. The other pop-ups did not substitute drug-drug interaction alerts, and were in addition to regular medication surveillance.

Our study has some potential strengths and limitations. We studied the effect in one hospital, and did not replicate our results in other hospitals. We made a comparison with a historical control group and there could be other reasons why the compliance changed over time, although we could not identify any potential alternative cause. For the selection of drugs that were included in the algorithm, we used the guideline developed by the Dutch Association of Hospital Pharmacists. However, whether drugs are included or not as for example a sodium lowering drugs is to some extent a matter of debate. Carboplatin and cisplatin do cause hyponatremia. However, sodium chloride is often given as supportive care, reducing the risk of hyponatremia. Other drugs that do cause hyponatremia, for example desmopressin, were not included in the guideline. We did not compare whether the acceptance rate with pop-up alerts was different with the acceptance rate in case patients lists were reviewed by a pharmacist and physicians were contacted by telephone. Since we found a high acceptance rate for the pop-up alerts, we are convinced that this method is more effective and less time-consuming. We did not study the time spend by physicians on managing the pop-up alerts and by pharmacists reviewing the overridden alerts. However, for methotrexate and bisphosphonates, only 81 alerts were shown to physicians in a 48 day period and we estimate that the total time spend will be less than an hour. For hyponatremia, a similar number of alerts was shown before and after implementation. Pharmacists had to review 132 overridden alerts, and made eight phone calls. We estimate that this took less than 3 h in 48 days. A strength of our study is that we implemented this clinical decision support system in daily routine, as we would for any other intervention. Therefore, the results are most likely similar if implemented in other hospitals.

To conclude, a clinical decision support system that alerts the physician during the medication ordering process is an effective way to improve the quality of prescribing. The three algorithms analyzed in this study resulted in a statistically significant increase in compliance with the recommendations.