Defining the Autonomy Gap: When Expectations Do Not Meet Reality in the Operating Room

Objective

To develop operative independence with essential procedures by the end of their training, residents need graded autonomy as they progress through training. This study compares autonomy expectations, as defined by faculty and residents, with autonomy measured in the operating room.

Methods

Operative procedures performed by general surgery residents between November 2012 and June 2013 were each assigned an autonomy score by the operating attending physician using a previously described rating scale (Zwisch). Scores range from minimum autonomy, “show and tell,” to maximum autonomy, “supervision only.” Autonomy expectations were defined by a survey asking faculty and residents what autonomy-level residents should achieve during each year of training for each of the 10 most commonly performed procedures. Faculty expectations, resident expectations, and actual operating room autonomy data were compared using analysis of variance with post hoc analysis by Tukey honestly significant difference test.

Results

A total of 1467 operative cases were scored using the Zwisch scale over the period of the study. The 10 most common procedures accounted for 56.3% (827) of the cases. Resident and faculty expectations of resident operative autonomy were similar. For only laparoscopic cholecystectomy, residents expected significantly more autonomy than the faculty did during the junior years but they agreed with the faculty for the chief year. When expectations were compared with actual performance, the resident autonomy level achieved was significantly less than that expected by residents or faculty or both for all 10 procedures in at least one postgraduate level. For every procedure performed more than 5 times during the study period by postgraduate years 3 to 5 residents, autonomy was significantly less than expected.

Conclusions

Surgical faculty and residents had similar expectations for resident operative autonomy, yet actual resident performance failed to achieve those shared expectations for even the most common procedures. This autonomy gap provides more evidence for concerns about the preparedness of graduating residents for independent practice.

Introduction

To optimize patient safety, it is incumbent on surgical residency training programs to graduate residents who are competent to independently care for patients at the completion of their training. Currently, competency is assessed by the American Board of Surgery by 3 methods: (a) written examination, (b) oral examination, and (c) summative assessment by the program director.¹ The written examination is designed to test primarily medical knowledge and decision making in the form of multiple-choice questions. The oral examination is designed to test some basic knowledge but primarily problem-solving skills and ability to make safe decisions in challenging situations. The summative assessment documents the resident’s ability to technically operate and care for patients in a real-world setting. The summative evaluation is usually based on rotation evaluations and personal experience with the resident and is a subjective summary of 5 years of clinical training in a few short paragraphs signed by the program director.

Objective data about the operations performed by residents during their training are currently obtained using the Accreditation Council on Graduate Medical Education’s electronic operative logs. From a list of more than 300 types of operative cases, surgical residents self-report the frequency with which they perform specific operations throughout their 5 years of training. Bell et al.² asked general surgery program directors to review this list of operations and designate those that are essential for practice in general surgery. The program directors selected 121 of these procedures as those which a resident should be able to perform independently by the completion of their training. The authors then compared this list to the number of actual procedures logged by graduating chief residents. Only 18 of the 121 procedures were performed more than 10 times during training by the average graduating resident. In fact, for 63 of the procedures, the most commonly reported frequency was zero. Even more important is that the operative logs include no indication of how much of the case the resident performed other than the broadly stated “significant portions of the procedure” or how much help from the attending surgeon was required for safe completion.

The first mandate for detailed assessment of specific procedural competency was introduced by the American Board of Surgery in 2012.³ The Board now requires each candidate for the qualifying examination to have written evaluations of 2 operative procedures performed by the program director or other faculty members over the course of their training. This is planned to increase to 6 cases for residents completing their training starting in the 2015 to 2016 academic year.³ However, this provides documented evidence for a small sample of the more than 750 operative cases a resident performs during training.

Bhatti and Cummings⁴ recommended a series of steps for continuous assessment of surgical competency that covered the continuum from resident selection to outcomes-based evaluation in practice. In addition to the standard written examinations and certification of judgmental competency by the program director and independent oral examiners, they noted a need to specifically evaluate procedural competency for specialty-specific index procedures. It is intuitive that, to determine if a resident is capable of safely performing an operation independently, they must be observed during independent operating experiences. In an era with multiple competing demands on academic faculty, including productivity requirements, pressures to improve operating room efficiency, and legal stressors, it is increasingly difficult to allow residents to work through procedures with limited assistance. Yet this elusive operative autonomy is not only important for evaluation purposes; procedural autonomy during training has been shown to be strongly associated with residents’ confidence in their abilities and even affects the type of procedures they plan to perform in practice.⁵

We have previously described a 4-step scale (the Zwisch scale) that can be used on a daily basis to document the amount of assistance required by a resident to successfully complete each case in the operating room.⁶ The amount of assistance required is an approximation of how capable the resident is to complete the operation autonomously. Operative autonomy is not an all-or-nothing event, the resident does not wake up one morning as an independent surgeon. Therefore, autonomy must be measured over time and in multiple different situations. To develop operative autonomy with essential procedures by the end of their training, residents need to achieve graded autonomy as they progress through training. As we begin to measure autonomy, for each level of training, it is important to establish progressive autonomy goals that are understood by faculty and residents. This study asks faculty and residents from a single institution to define goals for autonomy at each level of training for a selection of commonly performed general surgical procedures and compares these goals to actual autonomy measured in the operating room.