nach oben

Erschienen in:

06.03.2023 | original article

Resident evaluations: what are the predictors of future negative outcomes in surgical education?

verfasst von: Thomas Szabo Yamashita, MD, Sarah Lund, MD, Vicky J. -H. Yeh, PhD, Mariela Rivera, MD, Travis J. McKenzie, MD, John M. Stulak, MD, William S. Harmsen, Feven Abraha, Stephanie F. Heller, MD, James W. Jakub, MD

Erschienen in: European Surgery | Ausgabe 3-4/2023

Einloggen, um Zugang zu erhalten

Summary

Background

Evaluation variables that correlate with future negative outcomes in surgery allow for actionable steps and early intervention. Diverse evaluation systems have been used to quantify progression of residents. This study assessed our internal evaluation system to evaluate for variables that could predict negative outcomes in surgical education and, secondarily, whether differences in resident rating existed based on faculty–resident gender dyad.

Methods

This is a retrospective single-institution review of general surgery resident evaluations (2008–2020). Demographics, simulation lab, technical skill, and percentile for overall clinical performance are given by attendings. Logistic regression models were used to determine whether any factors were associated with negative resident outcomes and to determine if the faculty–resident gender dyad influenced evaluation ratings.

Results

A total of 3117 evaluations of 116 residents were included (66% male). Twenty negative outcomes were identified. In univariate logistic regression model American Board of Surgery In-Service Training Exam (ABSITE) scores, simulation lab scores and technical evaluations ≤ 20th percentile were associated with negative outcomes (p = 0.04). In multivariate analysis, no variables were associated with negative outcomes. Male staff tended to grade residents lower than female staff, independent of resident gender.

Conclusion

Although multiple subjective and objective variables, including national standardized tests, of surgical trainees are measured and reported, none of the variables collected over 10 years in 116 residents including over 3000 evaluations were independently associated with negative outcomes. This points to a need for development of new objective variables that can predict negative outcomes in residency. Preemptive identification of “at-risk” residents remains elusive.

Vorheriger Artikel Laparoscopic deroofing of simple liver cysts: do ancillary techniques, surgical devices, and indocyanine green improve outcomes?

Morris-Wiseman LF, Nfonsam VN. Early detection and remediation of problem learners. Surg Clin North Am. 2021;101(4):611–24. https://doi.org/10.1016/j.suc.2021.05.008.CrossRefPubMed

Gardner AK, AbdelFattah K. Comparison of simulation-based assessments and faculty ratings for general surgery resident milestone evaluation: are they telling the same story? Am J Surg. 2017;214(3):547–53. https://doi.org/10.1016/j.amjsurg.2016.07.025.CrossRefPubMed

Cassidy DJ, Chakraborty S, Panda N, et al. The surgical knowledge “growth curve”: predicting ABSITE scores and identifying “at-risk” residents. J Surg Educ. 2021;78(1):50–9. https://doi.org/10.1016/j.jsurg.2020.06.038.CrossRefPubMed

Gardner AK, Scott DJ. Repaying in kind: examination of the reciprocity effect in faculty and resident evaluations. J Surg Educ. 2016;73(6):e91–e4. https://doi.org/10.1016/j.jsurg.2016.04.015.CrossRefPubMed

Staats C. Understanding Implicit Bias. 6.

Moers F. Discretion and bias in performance evaluation: the impact of diversity and subjectivity. Account Organ Soc. 2005;30(1):67–80. https://doi.org/10.1016/j.aos.2003.11.001.CrossRef

Shellito AD, de Virgilio C, Lee G, et al. Investigating association between sex and faculty teaching evaluation in general surgery residency programs: a multi-institutional study. j Am Coll Surg. 2020;231(3):309–15e. https://doi.org/10.1016/j.jamcollsurg.2020.05.026.CrossRefPubMed

Dill-Macky A, Hsu CH, Neumayer LA, Nfonsam VN, Turner AP. The role of implicit bias in surgical resident evaluations. J Surg Educ. 2021; https://doi.org/10.1016/j.jsurg.2021.12.003.CrossRefPubMed

Rand VE, Hudes ES, Browner WS, Wachter RM, Avins AL. Effect of evaluator and resident gender on the American board of internal medicine evaluation scores. J Gen Intern Med. 1998;13(10):670–4. https://doi.org/10.1046/j.1525-1497.1998.00202.x.CrossRefPubMedPubMedCentral

10.

Angelo JL, Moazzez A, Neville A, Dauphine C, Lona Y, de Virgilio C. Investigating gender differences in faculty evaluations by trainees in a gender-balanced general surgery program. J Surg Educ. 2019;76(6):e132–e7. https://doi.org/10.1016/j.jsurg.2019.06.007.CrossRefPubMed

11.

Atesok K, Satava RM, Marsh JL, Hurwitz SR. Measuring surgical skills in simulation-based training. J Am Acad Orthop Surg. 2017;25(10):665–72. https://doi.org/10.5435/JAAOS-D-16-00253.CrossRefPubMed

12.

Sanfey H. Assessment of surgical training. Surgeon. 2014;12(6):350–6. https://doi.org/10.1016/j.surge.2014.07.001.CrossRefPubMed

13.

James OP, Robinson DBT, Hopkins L, et al. Strategic organisational skills predict surgical training success. Postgrad Med J. 2020; https://doi.org/10.1136/postgradmedj-2020-138763.CrossRefPubMed

14.

Deal SB, Scully RE, Wnuk G, George BC, Alseidi AA. Crowd-sourced and attending assessment of general surgery resident operative performance using global ratings scales. J Surg Educ. 2020;77(6):e214–e9. https://doi.org/10.1016/j.jsurg.2020.07.011.CrossRefPubMed

15.

Abdelsattar JM, AlJamal YN, Ruparel RK, Rowse PG, Heller SF, Farley DR. Correlation of objective assessment data with general surgery resident in-training evaluation reports and operative volumes. J Surg Educ. 2018;75(6):1430–6. https://doi.org/10.1016/j.jsurg.2018.04.016.CrossRefPubMed

16.

Ahle SL, Schuller M, Clark MJ, et al. Do end-of-rotation evaluations adequately assess readiness to operate? Acad Med. 2019;94(12):1946–52. https://doi.org/10.1097/ACM.0000000000002936.CrossRefPubMed

17.

Velez DR. Prospective factors that predict American board of surgery in-training examination performance: a systematic review. am Surg. 2021;87(12):1867–78. https://doi.org/10.1177/00031348211058626.CrossRefPubMed

18.

Gerull KM, Loe M, Seiler K, McAllister J, Salles A. Assessing gender bias in qualitative evaluations of surgical residents. Am J Surg. 2019;217(2):306–13. https://doi.org/10.1016/j.amjsurg.2018.09.029.CrossRefPubMed

19.

Meyerson SL, Odell DD, Zwischenberger JB, et al. The effect of gender on operative autonomy in general surgery residents. Surgery. 2019;166(5):738–43. https://doi.org/10.1016/j.surg.2019.06.006.CrossRefPubMed

20.

Brady JM, Bray A, Kim P, et al. Female residents give themselves lower scores than male colleagues and faculty evaluators on ACGME milestones. J Surg Educ. 2021;78(4):1305–11. https://doi.org/10.1016/j.jsurg.2020.12.003.CrossRefPubMed

21.

Skjold-Ødegaard B, Ersdal HL, Assmus J, Soreide K. Comparison of performance score for female and male residents in general surgery doing supervised real-life laparoscopic appendectomy: is there a norse shield-maiden effect? World J Surg. 2021;45(4):997–1005. https://doi.org/10.1007/s00268-020-05921-4.CrossRefPubMedPubMedCentral

Titel: Resident evaluations: what are the predictors of future negative outcomes in surgical education?
verfasst von: Thomas Szabo Yamashita, MD
Sarah Lund, MD
Vicky J. -H. Yeh, PhD
Mariela Rivera, MD
Travis J. McKenzie, MD
John M. Stulak, MD
William S. Harmsen
Feven Abraha
Stephanie F. Heller, MD
James W. Jakub, MD
Publikationsdatum: 06.03.2023
Verlag: Springer Vienna
Erschienen in: European Surgery / Ausgabe 3-4/2023
Print ISSN: 1682-8631
Elektronische ISSN: 1682-4016
DOI: https://doi.org/10.1007/s10353-023-00797-9

Springer Medizin Österreich

Summary

Background

Methods

Results

Conclusion

Bitte loggen Sie sich ein, um Zugang zu diesem Inhalt zu erhalten

Weitere Artikel der Ausgabe 3-4/2023

The role of surgical simulation in future training

Training in vascular trauma surgery for non-vascular surgeons

Surgery orients positivity

Surgical management of colorectal liver metastases—a practical clinical approach

Remote training and evaluation of a simulator-based training course for complex endovascular procedures

Laparoscopic deroofing of simple liver cysts: do ancillary techniques, surgical devices, and indocyanine green improve outcomes?