Comparison of simulation-based assessments and faculty ratings for general surgery resident milestone evaluation: Are they telling the same story?
Section snippets
Methods
Participants consisted of PGY1 trainees in the UT Southwestern General Surgery Residency program. The IRB deemed this project exempt. All training sessions took place in the last 2 months of Internship. Participants received pre training materials 1 week before attending the simulation session. Pretraining materials consisted of the SPIKES Delivering Bad News protocol,5 a surgical safety checklist from the World Health Organization (WHO),6 the Society of American Gastroenterologists and
Results
Twenty-six PGY1 residents participated in the simulation training.
Comments
This study was designed to examine the extent to which simulation can fulfill specific aspects of the milestone project's goals for resident competency. Our work suggests that simulated environments can be used to create a standardized avenue for assessing resident competency in these specific areas. In addition, these results suggest that simulation may provide unique data that is otherwise unobtainable from clinical assessments. Our study demonstrates that faculty may not have enough
Conclusions
Given the level of variability of competence among graduating general surgery residents,17, 18, 19, 20 methods to assess resident competence early and often and in a standardized fashion should be thoroughly developed and implemented across training programs. One of the goals of the milestones project was to direct and motivate programs toward this end. However, there is still confusion among programs regarding the best way to implement these milestones assessments,21, 22 as well as which
References (22)
- et al.
Gearing up for milestones in surgery: will simulation play a role?
Surgery
(2015) - et al.
Best practices across surgical specialties relating to simulation-based training
Surgery
(2015) - et al.
Developing a comprehensive resident education evaluation system in the era of milestone assessment
J Surg Educ
(2015) - et al.
Residents seeking informed consent: are they adequately knowledgeable?
Curr Surg
(2002) - et al.
Construct and face validity of the American College of Surgeons/Association of Program Directors in Surgery laparoscopic troubleshooting team training exercise
Am J Surg
(2012) - et al.
Frame-of-reference training for simulation-based intraoperative communication assessment
Am J Surg
(2016) - et al.
Informed consent training improves surgery resident performance in simulated encounters with standardized patients
Am J Surg
(2015) - et al.
What is the future of training in surgery? Needs assessment of national stakeholders
Surgery
(2014) - The General Surgery Milestone Project. Available at:...
- et al.
SPIKES-A six-step protocol for delivering bad news: application to the patient with cancer
Oncologist
(2000)
Multidisciplinary crisis simulations: the way forward for training surgical teams
World J Surg
Cited by (9)
The Lake Wobegon effect is real: All general surgery residents appear to be better than average
2022, Surgery in Practice and ScienceJunior doctor experiences and challenges in obtaining surgical informed consent: A qualitative systematic review & meta-ethnography
2021, Journal of Surgical ResearchCitation Excerpt :Few junior doctors have had the opportunity to acquire SIC under the direct supervision of a more senior doctor, with 73.3% of the interns surveyed by Heaney et al reporting they had never had such an opportunity.29 Thirteen of the thirty-four included studies recommend that additional educational interventions are implemented for all staff, including junior doctors, to improve skills and knowledge around SIC.8,19,27-31,36,41,43-46 Suggested interventions include assessment of junior doctors’ consenting skills in simulated environments22,42,46, bioethics and communication for junior doctors23,37, dedicated education programs for junior doctors to acquire SIC22; spending more time with senior doctors observing juniors’ skills in consenting, allowing juniors to have more theatre time to observe procedures29,31, ensuring surgical trainees have adequate understanding of the details of surgical procedures37, direct observation by senior doctors while acquiring SIC44,23,31,48, use of an online learning module44, use of required reading44, and ensuring consent is taught adequately in the medical school curriculum[31], ensuring trainees have access to further general didactic education on SIC.23,28,30,41
International Medical Graduates are Comparable to American Medical Graduates as General Surgery Interns
2021, Journal of Surgical ResearchCitation Excerpt :Although several programs have developed innovative ways to assess their trainees, there is a paucity of standardized assessments that accurately map a resident's trajectory in terms of surgical skill. Gardner et al.33 found no significant correlation between simulation-based assessments and end-of-rotation faculty evaluations and identified that the latter could not accurately distinguish performance toward the set clinical competencies. In another article from the same institution, the authors proposed a comprehensive system that combined simulation, ABSITE scores, and subjective evaluations among others to critically analyze resident progress.34
What do quantitative ratings and qualitative comments tell us about general surgery residents’ progress toward independent practice? Evidence from a 5-year longitudinal cohort
2019, American Journal of SurgeryCitation Excerpt :Prior studies have noted concerns over rotation evaluation scores as not being reliable assessments of learner performance, despite their prevalent use in making promotion decisions.8–13 However, recent studies have shown that rotation evaluations scores can be reliable and demonstrate validity evidence, provided that evaluations from multiple raters are collected and aggregated over a sufficient period of time.7,14–21 This study focuses on synthesizing both quantitative and qualitative data from rotation evaluations, in the context of identifying competency-based developmental levels.
Emotional Intelligence and Delivering Bad News: The Jury is Still Out
2019, Journal of Surgical EducationCitation Excerpt :All phases of the training session were video recorded for performance assessment. Performance for DBN was evaluated using a 10-item checklist tool (achieved/did not achieve) internally developed based on best practice suggestions by Baile etal.,19 and used elsewhere.24 The checklist for DBN was completed at the completion of all simulation sessions via video review by 2 simulation faculty members (surgeon and PhD).
Resident evaluations: what are the predictors of future negative outcomes in surgical education?
2023, European Surgery - Acta Chirurgica Austriaca
There were no relevant financial relationships or any sources of support in the form of grants, equipment, or drugs.
The authors report no conflicts of interest.