Skip to main content

Advertisement

SpringerLink
Log in

How well do selection tools predict performance later in a medical programme?

  • Published:
Advances in Health Sciences Education Aims and scope Submit manuscript

A Commentary to this article was published on 27 June 2012

Abstract

The choice of tools with which to select medical students is complex and controversial. This study aimed to identify the extent to which scores on each of three admission tools (Admission GPA, UMAT and structured interview) predicted the outcomes of the first major clinical year (Y4) of a 6 year medical programme. Data from three student cohorts (n = 324) were analysed using regression analyses. The Admission GPA was the best predictor of academic achievement in years 2 and 3 with regression coefficients (B) of 1.31 and 0.9 respectively (each P < 0.001). Furthermore, Admission GPA predicted whether or not a student was likely to earn ‘Distinction’ rather than ‘Pass’ in year 4. In comparison, UMAT and interview showed low predictive ability for any outcomes. Interview scores correlated negatively with those on the other tools. None of the tools predicted failure to complete year 4 on time, but only 3% of students fell into this category. Prior academic achievement remains the best measure of subsequent student achievement within a medical programme. Interview scores have little predictive value. Future directions include longer term studies of what UMAT predicts, and of novel ways to combine selection tools to achieve the optimum student cohort.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Australian Medical Council. (2009). Assessment and accreditation of medical schools: standards and procedures. Retrieved from http://www.amc.org.au/images/Medschool/procedures%20medical%20schools%202009.pdf.

  • Arnold, H. J., Feldman, D. C., & Purbhoo, M. (1985). The role of social-desirability response bias in turnover research. Academy of Management Journal, 28(4), 955–966.

    Article  Google Scholar 

  • Australian Council for Educational Research. (2011). UMAT Retrieved March 10, 2011, from http://umat.acer.edu.au/index.php.

  • Boelen, C., & Woollard, B. (2009). Social accountability and accreditation: A new frontier for educational institutions. Medical Education, 43(9), 887–894.

    Article  Google Scholar 

  • Bore, M., Munro, D., & Powis, D. (2009). A comprehensive model for the selection of medical students. Medical Teacher, 31(12), 1066–1072.

    Article  Google Scholar 

  • Callahan, C., Hojat, M., Veloski, J., Erdmann, J. B., & Gonnella, J. S. (2010). The predictive validity of three versions of the MCAT in relation to performance in medical school, residency, and licensing examinations: A longitudinal study of 36 classes of Jefferson Medical College. Academic Medicine, 85(6), 980.

    Article  Google Scholar 

  • Carr, S. E. (2009). Emotional intelligence in medical students: Does it correlate with selection measures? Medical Education, 43(11), 1069–1077.

    Article  Google Scholar 

  • Collins, J., Jones, J., & White, G. (1993). Demographic variables in Auckland medical students. The New Zealand Medical Journal, 106(960), 306.

    Google Scholar 

  • Donnon, T., & Paolucci, E. O. (2008). A generalizability study of the medical judgment vignettes interview to assess students’ noncognitive attributes for medical school. BMC Medical Education, 8(1), 58.

    Article  Google Scholar 

  • Donnon, T., Paolucci, E. O., & Violato, C. (2007). The predictive validity of the MCAT for medical school performance and medical board licensing examinations: A meta-analysis of the published research. Academic Medicine, 82(1), 100.

    Article  Google Scholar 

  • Dore, K. L., Reiter, H. I., Eva, K. W., Krueger, S., Scriven, E., Siu, E., et al. (2009). Extending the interview to all medical school Candidates-Computer-Based Multiple Sample Evaluation of Noncognitive Skills (CMSENS). Academic Medicine, 84(10), S9.

    Article  Google Scholar 

  • Dubovsky, S. L., Gendel, M. H., Dubovsky, A. N., Levin, R., Rosse, J., & House, R. (2008). Can admissions interviews predict performance in residency? Academic Psychiatry, 32(6), 498.

    Article  Google Scholar 

  • Edwards, J. C., Elam, C. L., & Wagoner, N. E. (2001). An admission model for medical schools. Academic Medicine, 76(12), 1207–1212.

    Article  Google Scholar 

  • Eva, K. W., Reiter, H. I., Trinh, K., Wasi, P., Rosenfeld, J., & Norman, G. R. (2009). Predictive validity of the multiple mini interview for selecting medical trainees. Medical Education, 43(8), 767–775.

    Article  Google Scholar 

  • Ferguson, E., James, D., & Madeley, L. (2002). Factors associated with success in medical school: Systematic review of the literature. British Medical Journal, 324, 952–957.

    Article  Google Scholar 

  • Gorman, D., Monigatti, J., & Poole, P. (2008). On the case for an interview in medical student selection. Internal Medicine Journal, 38(8), 621–623.

    Article  Google Scholar 

  • James, D., Yates, J., & Nicholson, S. (2010). Comparison of A level and UKCAT performance in students applying to UK medical and dental schools in 2006: Cohort study. British Medical Journal, 340(feb16 1), c478.

  • Koc, T., Katona, C., & Rees, P. J. (2008). Contribution of medical students to admission interviews. Medical Education, 42(3), 315–321.

    Article  Google Scholar 

  • Kreiter, C. D., Yin, P., Solow, C., & Brennan, R. L. (2004). Investigating the reliability of the medical school admissions interview. Advances in Health Sciences Education, 9(2), 147–159.

    Article  Google Scholar 

  • Lambe, P., & Bristow, D. (2011). Predicting medical student performance from attributes at entry: A latent class analysis. Medical Education, 45(3), 308–316.

    Article  Google Scholar 

  • Lumb, A. B., Homer, M., & Miller, A. (2010). Equity in interviews: Do personal characteristics impact on admission interview scores? Medical Education, 44(11), 1077–1083. doi:10.1111/j.1365-2923.2010.03771.x.

    Article  Google Scholar 

  • Lynch, B., MacKenzie, R., Dowell, J., Cleland, J., & Prescott, G. (2009). Does the UKCAT predict year 1 performance in medical school? Medical Education, 43(12), 1203–1209.

    Article  Google Scholar 

  • McManus, I., Smithers, E., Partridge, P., Keeling, A., & Fleming, P. (2003). A levels and intelligence as predictors of medical careers in UK doctors: 20 year prospective study. BMJ, 327(7407), 139–142. doi:10.1136/bmj.327.7407.139.

    Article  Google Scholar 

  • Mercer, A. (2007). Selecting medical students: An Australian case study. PhD, Murdoch University.

  • Mercer, A., & Chiavaroli, N. (2007). UMAT: A validity study. Perth: University of Western Australia.

    Google Scholar 

  • Mercer, A., & Puddey, I. (2011). Admission selection criteria as predictors of outcomes in an undergraduate medical course: A prospective study. Medical Teacher, 1–8. doi:10.3109/0142159X.2011.577123.

  • Ministry of Health. (2006). Training the medical workforce 2006 and beyond. Wellington, New Zealand: Ministry of Health.

    Google Scholar 

  • Murden, R., Galloway, G. M., Reid, J. C., & Colwill, J. M. (1978). Academic and personal predictors of clinical success in medical school. Academic Medicine, 53(9), 711.

    Article  Google Scholar 

  • Norman, G. (2004). Editorial—the morality of medical school admissions. Advances in Health Sciences Education, 9(2), 79–82.

    Article  Google Scholar 

  • O’Neill, L., Wallstedt, B., Eika, B., & Hartvigsen, J. (2011). Factors associated with dropout in medical education: A literature review. Medical Education, 45(5), 440–454. doi:10.1111/j.1365-2923.2010.03898.x.

    Article  Google Scholar 

  • Pekrun, R., Elliot, A. J., & Maier, M. A. (2009). Achievement goals and achievement emotions: Testing a model of their joint relations with academic performance. Journal of Educational Psychology, 101(1), 115–135.

    Article  Google Scholar 

  • Poole, P. (2010). Medical workforce development in New Zealand: Insights from a medical programme. MD, University of Auckland, Auckland. Retrieved from http://hdl.handle.net/2292/6667.

  • Poole, P., Moriarty, H., Wearn, A., Wilkinson, T., & Weller, J. (2009). Medical student selection in New Zealand: Looking to the future. The New Zealand medical journal, 122(1306), 88.

    Google Scholar 

  • Poole, P., Shulruf, B., Rudland, J., & Wilkinson, T. (in press). Comparison of UMAT and admission GPA on The Prediction of Performance on Medical School Assessments: A National, Cross-Institution Study. Medical Education.

  • Powis, D. (1998). How to do it: Select medical students. BMJ: British Medical Journal, 317(7166), 1149.

    Article  Google Scholar 

  • Prideaux, D., Roberts, C., Eva, K., Centeno, A., Mccrorie, P., Mcmanus, C., et al. (2011). Assessment for selection for the health care professions and specialty training: Consensus statement and recommendations from the Ottawa 2010 conference. Medical Teacher, 33, 215–223.

    Article  Google Scholar 

  • Salvatori, P. (2001). Reliability and validity of admissions tools used to select students for the health professions. Advances in Health Sciences Education, 6(2), 159–175.

    Article  Google Scholar 

  • Shulruf, B. (2008). A dual admission policy: Enhancing equal opportunities in higher education by merit-based admission policy. Paper presented at the hurst seminar on higher education and equality of opportunity: Cross-national perspectives, Ben Gurion University of the Negev, Beer Sheva, Israel, 3–5 June 2008.

  • Shulruf, B., Hattie, J., & Tumen, S. (2008). The predictability of enrolment and first year university results from secondary school performance. Studies in Higher Education, 33(6), 685–698.

    Google Scholar 

  • Shulruf, B., Turner, R., & Hattie, J. (2009). A dual admission model for equity in higher education: A multi-cohort longitudinal study. Procedia-Social and Behavioral Sciences, 1(1), 2416–2420.

    Article  Google Scholar 

  • Simpson, K., & Budd, K. (1996). Medical student attrition: A 10-year survey in one medical school. Medical Education, 30(3), 172–178. doi:10.1111/j.1365-2923.1996.tb00739.x.

    Article  Google Scholar 

  • Streyffeler, L., Altmaier, E. M., Kuperman, S., & Patrick, L. E. (2009). Development of a medical school admissions interview phase 2: Predictive validity of cognitive and non-cognitive attributes. Medical Education Online, 10, 1–5.

    Google Scholar 

  • Urlings-Strop, L. C. (2009). Selection of medical students: A controlled experiment. Medical Education (43), 175–183.

  • Walsh, K. (2010). Cost effectiveness in medical education. Oxford: Radcliffe Publishing Ltd.

    Google Scholar 

  • Westwood, M. A., Nunn, L. M., Redpath, C., Mills, P., & Crake, T. (2008). Applicants regard structured interviews as a fair method of selection: An audit of candidates. Journal of the Royal Society of Medicine, 101(5), 252.

    Article  Google Scholar 

  • White, C., Dey, E., & Fantone, J. (2009). Analysis of factors that predict clinical performance in medical school. Advances in Health Sciences Education, 14(4), 455–464. doi:10.1007/s10459-007-9088-9.

    Article  Google Scholar 

  • Wilkinson, D., Zhang, J., Byrne, G. R., Luke, H., Ozolins, I., Parker, M. E., et al. (2008). Medical school selection criteria and the prediction of academic performance evidence leading to change in policy and practice at the University of Queensland. Medical Journal of Australia, 188(6), 349–354.

    Google Scholar 

  • Wilkinson, D., Zhang, J., & Parker, M. (2011). Predictive validity of the undergraduate medicine and health sciences admission test for medical students’ academic performance. The Medical Journal of Australia, 194(7), 341–344.

    Google Scholar 

  • Wright, S. R., & Bradley, P. M. (2010). Has the UK clinical aptitude test improved medical student selection? Medical Education, 44(11), 1069–1076. doi:10.1111/j.1365-2923.2010.03792.x.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Centre for Medical and Health Sciences Education, Faculty of Medical and Health Sciences, The University of Auckland, Private Bag 92019, Auckland Mail Centre, Auckland, 1142, New Zealand

    Boaz Shulruf

  2. Department of Medicine, Faculty of Medical and Health Sciences, The University of Auckland, Private Bag 92019, Auckland Mail Centre, Auckland, 1142, New Zealand

    Phillippa Poole

  3. Faculty of Medical and Health Sciences, The University of Auckland, Private Bag 92019, Auckland Mail Centre, Auckland, 1142, New Zealand

    Grace Ying Wang

  4. Education Unit, Faculty of Medicine, The University of Otago, Dunedin, New Zealand

    Joy Rudland

  5. Faculty of Medicine, The University of Otago, Christchurch, New Zealand

    Tim Wilkinson

Authors
  1. Boaz Shulruf
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Phillippa Poole
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Grace Ying Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Joy Rudland
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tim Wilkinson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Boaz Shulruf.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Shulruf, B., Poole, P., Wang, G.Y. et al. How well do selection tools predict performance later in a medical programme?. Adv in Health Sci Educ 17, 615–626 (2012). https://doi.org/10.1007/s10459-011-9324-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10459-011-9324-1

Keywords

Search

Navigation