Improving mortality of coronary surgery over first four years of independent practice: retrospective examination of prospectively collected data from 15 surgeons

Design of the project

We addressed four specific questions:

• Are there differences in observed and predicted mortality between patients operated on by newly appointed surgeons and their more experienced colleagues?

• Are there changes in observed and predicted mortality in patients operated on by newly appointed surgeons over the first four years of appointment?

• If there are improvements in risk adjusted mortality over the first few years of appointment, what is the time course of those changes?

• If there are decreases in mortality, are they seen in low or high risk patients?

The north west regional audit project started in 1997. Since that time there have been 13 new appointments. In addition, two surgeons practising in 1997 were in the first four years of their consultant appointment. For the analysis we pooled surgeons into four groups according to year of independent practice (first, second, third, and fourth years). We included all temporary appointments of less than a year as first year appointments. Associate specialists who had been in independent practice for more than four years by the time of their consultant appointment were not included as new appointments; if they had been in independent practice for less than four years they were included in the appropriate year as a new consultant. We also analysed data excluding locum appointments from the dataset.

Statistical analysis

Categorical data are shown as percentages with 95% confidence intervals, while continuous data are shown as means. We used χ² tests for trend and Wilcoxon rank sum tests as appropriate to test for significant differences between groups. Crude mortality was determined for each patient and a predicted mortality calculated by using additive EuroSCORE.4 If a factor necessary to calculate the EuroSCORE was missing in the record, that factor was assumed to be absent (this occurred in less than 2% of cases). While the mean EuroSCORE is often given as an indication of operative risk of a group, the score is not normally distributed and so we have given results as means and medians with 25th and 75th centiles. To investigate in which type of patients the observed changes occurred, we subdivided patients into low (EuroSCORE < 6) and high (EuroSCORE < 6) risk groups for further analysis.2 To adjust for a potential time effect we constructed a logistic regression model on all cases (n = 18 913) that included the financial year of the operation (treated as 1997-8 = 1, 1998-9 = 2, etc) and the EuroSCORE.5 Using the logistic model, we then calculated risk adjusted mortality percentages for the first four years of independent practice, in line with the system developed in the state of New York.6 The results reflect performance adjusted for time and case mix.

Principal findings

We have shown significant improvements in crude and risk adjusted mortality associated with coronary artery surgery over the four years after surgeons are appointed to independent practice. The observed mortality for surgeons in their first and second years after appointment was slightly higher than for established consultants but lower in the third and fourth years. These improvements were due to better outcomes in patients undergoing low risk surgery.

Strengths and weaknesses of the study

Our study has been conducted on a dataset produced from all patients undergoing cardiac surgery in north west England since 1997. The data have been subjected to local validation and have the confidence of clinicians but have not been subjected to external validation, which is a weakness. We consider that the number of surgeons, the large population of patients, and the fact that these appointments were across four different hospitals add strength to our study. We were initially concerned that some of the learning curve effect detected may have been due to overall changes in mortality because of progressive improvements in outcome with time, which is an effect that has been clearly seen in previous work.7 The improvements we have shown over the first four years of practice were still significant after we adjusted for time and case mix by logistic regression. We have looked only at mortality and not at morbidity, which may be a more sensitive indicator of subtle improvements in the quality of care.

We observed a significant decrease in crude mortality as surgeons become more experienced. We used a well accepted model of risk prediction, the additive EuroSCORE,4 to look for changes in case mix to see if alterations in predicted mortality might explain the observed mortality. The EuroSCORE is generally accepted to be a good but not perfect predictor of mortality.2 8 9 Predicted mortality increased in the four years after appointment, suggesting that surgeons are operating on patients with increasing comorbidity, which confirms local clinical perceptions. Use of the ratio of observed to expected mortality as an overall indicator of risk adjusted quality of care highlights this effect of decreasing mortality despite increasing complexity of case mix. While there must be some caution exercised when using risk prediction models, it would seem to be appropriate in this setting, but it is conceivable that more experienced surgeons may become “smarter” at exploiting limitations within the risk prediction model, which would not affect our observations on crude mortality but may potentially confound the risk adjusted analysis.

The mortality in patients of surgeons in the first two years of appointment was slightly higher than that in patients of more established consultants performing surgery during the time period studied. The observation that mortality in patients of surgeons in their third and fourth years of appointment is below that seen in patients of more established surgeons (both crude mortality and risk adjusted outcomes), suggests that either the performance of these consultants is better than their peers, or that there are complex issues associated with case mix and higher risk surgery that may account for these differences but were not detected with our methods.

Comparison with other studies

There are few studies on learning curves in surgery. There have been reports of learning curve effects with new techniques or with new complex operations in both cardiac10 11 and non-cardiac surgery.12^–17 There are thought to be three separate components of the learning curve for new procedures: the specialist community, the institution, and the individual surgeon.11 We examined the effects of the learning curve in surgeons on a well established procedure for which they have been trained. Most trainees will have performed many hundreds of coronary artery bypass operations before appointment in hospitals with existing skills and infrastructure, so we are not looking at aspects of learning curves of the specialist community or institution. We saw significant improvements in risk adjusted mortality in the four years after appointment, which is an “individual surgeon” effect. We are not aware of previous reports on learning curve effects associated with moving from training grade to independent practice.

A recent study on learning curve effects in complex laparoscopic colorectal surgery14 suggests a steady state of technical competence is achieved after 70-80 procedures. The number of operations needed to achieve the “plateau phase” for technical issues in coronary artery surgery is probably similar. As all new appointees in this study would have performed well over 100 operations, while there may still be ongoing technical improvements after appointment, the improvements we have detected are probably related to issues other than primary technical ability.

Implications

If the improvements seen after appointment are not related to technical ability it implies that surgeons improve the non-technical aspects of surgery, including decisions on case selection and aspects of preoperative, intraoperative, and postoperative care and decision making. Our study suggests that patients and hospitals can be reassured that mortality is not higher in patients of newly appointed surgeons, but initiatives that may modify the learning curve will have advantages for patients, hospitals, and surgeons.

There are several possible ways to minimise the effects of learning curves: improve training, increase supervision of new appointments, and introduce other strategies to accelerate improvement. The effect seen here was detected in surgeons who have been trained under existing structures (the tiered registrar/senior registrar system or seamless Calman training) and will have spent in excess of five years as specialist trainees. Plans to shorten the surgical training period, along with a reduction in training hours due to implementation of the European working time directive, may have implications on the experience of new consultants in the future, which would act to increase possible learning curve effects unless other modifying influences are introduced.

It has been suggested that learning curve effects in medicine are similar to those in manufacturing17 and that by applying learning curve theory, higher quality health care may be achieved. Learning curve theories accepted in manufacturing are that outcomes improve with higher volume due to learning, increases in improvement become smaller with progressive experience, and increases in improvement follow a predictable pattern. In previous studies of learning curves in cardiac10 11 and non-cardiac12^–17 surgery, effects have all been detected on a relatively small number of cases. We have observed a learning curve effect in surgeons who are performing high volumes of surgery. While we saw continuous improvements in each of the first four years of appointment, the most marked improvements were between the third and fourth years, which is at odds with data from manufacturing. The annual volumes of coronary artery surgery in units and individuals are high, which differs from the typical small number of cases undertaken in other surgical specialties.

Volume of experience, incentives, and managerial factors are potential modifying influences on the learning curve.18 It seems inappropriate to increase the volume of surgery to higher than current levels. Planned publication of surgeon specific mortality and use of performance data in clinical excellence awards may well influence learning curves in future but may also encourage newly appointed surgeons to turn down higher risk cases, unless robust risk adjusted measures are used for performance management. Managerial input—including the use of clinical pathways, robust clinical risk management strategies, effective supervision, and targeted continuing professional development—would seem to be the most effective way to accelerate learning curve effects. Experienced clinical support for new appointments, not just for technical issues but for all aspects of clinical practice, including case selection and perioperative management, seems sensible.

Unanswered questions and future research

Our study raises several questions that need to be addressed in future studies. We do not know whether causes of death change as surgeons become more experienced, which may help to guide plans to improve outcomes in newly appointed surgeons. While the effect for cardiac surgery seems to be marked, we do not know whether the learning curve is restricted to this specialty or whether it is seen in other surgical specialties or is a widespread phenomenon throughout medicine. There are several initiatives that are leading to a reduction in hours during the working week and limiting the number of years of training for surgeons, and we do not know if this will make the effects we have seen more marked. Finally, we do not know whether initiatives such as a structured supervision and targeted continuous professional development would accelerate the learning curve effect.