Skip to main content

Advertisement

SpringerLink
Log in

A simple scoring system for breast MRI interpretation: does it compensate for reader experience?

  • Breast
  • Published:
European Radiology Aims and scope Submit manuscript

Abstract

Purpose

To investigate the impact of a scoring system (Tree) on inter-reader agreement and diagnostic performance in breast MRI reading.

Materials and methods

This IRB-approved, single-centre study included 100 patients with 121 consecutive histopathologically verified lesions (52 malignant, 68 benign). Four breast radiologists with different levels of MRI experience and blinded to histopathology retrospectively evaluated all examinations. Readers independently applied two methods to classify breast lesions: BI-RADS and Tree. BI-RADS provides a reporting lexicon that is empirically translated into likelihoods of malignancy; Tree is a scoring system that results in a diagnostic category. Readings were compared by ROC analysis and kappa statistics.

Results

Inter-reader agreement was substantial to almost perfect (kappa: 0.643–0.896) for Tree and moderate (kappa: 0.455–0.657) for BI-RADS. Diagnostic performance using Tree (AUC: 0.889–0.943) was similar to BI-RADS (AUC: 0.872–0.953). Less experienced radiologists achieved AUC: improvements up to 4.7 % using Tree (P-values: 0.042–0.698); an expert’s performance did not change (P = 0.526). The least experienced reader improved in specificity using Tree (16 %, P = 0.001). No further sensitivity and specificity differences were found (P > 0.1).

Conclusion

The Tree scoring system improves inter-reader agreement and achieves a diagnostic performance similar to that of BI-RADS. Less experienced radiologists, in particular, benefit from Tree.

Key Points

• The Tree scoring system shows high diagnostic accuracy in mass and non-mass lesions.

• The Tree scoring system reduces inter-reader variability related to reader experience.

• The Tree scoring system improves diagnostic accuracy in non-expert readers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Warner E, Messersmith H, Causer P et al (2008) Systematic review: using magnetic resonance imaging to screen women at high risk for breast cancer. Ann Intern Med 148:671–679

    Article  PubMed  Google Scholar 

  2. Riedl CC, Luft N, Bernhart C et al (2015) Triple-modality screening trial for familial breast cancer underlines the importance of magnetic resonance imaging and questions the role of mammography and ultrasound regardless of patient mutation status, age, and breast density. J Clin Oncol Off J Am Soc Clin Oncol. doi:10.1200/JCO.2014.56.8626

    Google Scholar 

  3. Houssami N, Ciatto S, Macaskill P et al (2008) Accuracy and surgical impact of magnetic resonance imaging in breast cancer staging: systematic review and meta-analysis in detection of multifocal and multicentric cancer. J Clin Oncol Off J Am Soc Clin Oncol 26:3248–3258

    Article  Google Scholar 

  4. Kuhl C (2007) The current status of breast MR imaging. Part I. Choice of technique, image interpretation, diagnostic accuracy, and transfer to clinical practice. Radiology 244:356–378

    Article  PubMed  Google Scholar 

  5. Sardanelli F, Boetes C, Borisch B et al (2010) Magnetic resonance imaging of the breast: recommendations from the EUSOMA working group. Eur J Cancer Oxf Engl 46:1296–1316

    Article  Google Scholar 

  6. Mann RM, Balleyguier C, Baltzer PA, European Society of Breast Imaging (EUSOBI), with language review by Europa Donna–The European Breast Cancer Coalition et al (2015) Breast MRI: EUSOBI recommendations for women’s information. Eur Radiol. doi:10.1007/s00330-015-3807-z

    PubMed Central  Google Scholar 

  7. The American College of Radiology (ACR) (2013) Breast Imaging Reporting and Data System Atlas (BI-RADS® Atlas), Reston, VA

  8. Pinker K, Bogner W, Baltzer P et al (2014) Improved differentiation of benign and malignant breast tumors with multiparametric 18Fluorodeoxyglucose positron emission tomography magnetic resonance imaging: a feasibility study. Clin Cancer Res. doi:10.1158/1078-0432.CCR-13-2810

    PubMed  Google Scholar 

  9. Benndorf M, Baltzer PAT, Kaiser WA (2011) Assessing the degree of collinearity among the lesion features of the MRI BI-RADS lexicon. Eur J Radiol 80:e322–e324

    Article  PubMed  Google Scholar 

  10. Ikeda DM, Hylton NM, Kinkel K et al (2001) Development, standardization, and testing of a lexicon for reporting contrast-enhanced breast magnetic resonance imaging studies. J Magn Reson Imaging 13:889–895

    Article  CAS  PubMed  Google Scholar 

  11. Kim SJ, Morris EA, Liberman L et al (2001) Observer variability and applicability of BI-RADS terminology for breast MR imaging: invasive carcinomas as focal masses. AJR Am J Roentgenol 177:551–557

    Article  CAS  PubMed  Google Scholar 

  12. Kinkel K, Helbich TH, Esserman LJ et al (2000) Dynamic high-spatial-resolution MR imaging of suspicious breast lesions: diagnostic criteria and interobserver variability. AJR Am J Roentgenol 175:35–43

    Article  CAS  PubMed  Google Scholar 

  13. Stoutjesdijk MJ, Fütterer JJ, Boetes C et al (2005) Variability in the description of morphologic and contrast enhancement characteristics of breast lesions on magnetic resonance imaging. Invest Radiol 40:355–362

    Article  PubMed  Google Scholar 

  14. OCEBM Levels of Evidence Working Group. “The Oxford Levels of Evidence 2”. Oxford Centre for Evidence-Based Medicine. http://www.cebm.net/index.aspx?o=5653

  15. Baltzer PAT, Dietzel M, Kaiser WA (2013) A simple and robust classification tree for differentiation between benign and malignant lesions in MR-mammography. Eur Radiol 23:2051–2060

    Article  PubMed  Google Scholar 

  16. Baum F, Fischer U, Vosshenrich R, Grabbe E (2002) Classification of hypervascularized lesions in CE MR imaging of the breast. Eur Radiol 12:1087–1092

    Article  CAS  PubMed  Google Scholar 

  17. Nunes LW, Schnall MD, Orel SG et al (1997) Breast MR imaging: interpretation model. Radiology 202:833–841

    Article  CAS  PubMed  Google Scholar 

  18. Tozaki M, Igarashi T, Matsushima S, Fukuda K (2005) High-spatial-resolution MR imaging of focal breast masses: interpretation model based on kinetic and morphological parameters. Radiat Med 23:43–50

    PubMed  Google Scholar 

  19. Tozaki M, Fukuda K (2006) High-spatial-resolution MRI of non-masslike breast lesions: interpretation model based on BI-RADS MRI descriptors. AJR Am J Roentgenol 187:330–337

    Article  PubMed  Google Scholar 

  20. Demartini WB, Kurland BF, Gutierrez RL et al (2011) Probability of malignancy for lesions detected on breast MRI: a predictive model incorporating BI-RADS imaging features and patient characteristics. Eur Radiol 21:1609–1617

    Article  PubMed  Google Scholar 

  21. Pinker K, Bogner W, Baltzer P et al (2014) Improved diagnostic accuracy with multiparametric magnetic resonance imaging of the breast using dynamic contrast-enhanced magnetic resonance imaging, diffusion-weighted imaging, and 3-dimensional proton magnetic resonance spectroscopic imaging. Invest Radiol. doi:10.1097/RLI.0000000000000029

    Google Scholar 

  22. Perry N, Broeders M, de Wolf C et al (2008) European guidelines for quality assurance in breast cancer screening and diagnosis. Fourth edition—summary document. Ann Oncol 19:614–622

    Article  CAS  PubMed  Google Scholar 

  23. Gutierrez RL, DeMartini WB, Eby PR et al (2009) BI-RADS lesion characteristics predict likelihood of malignancy in breast MRI for masses but not for nonmasslike enhancement. AJR Am J Roentgenol 193:994–1000

    Article  PubMed  Google Scholar 

  24. Baltzer PAT, Benndorf M, Dietzel M et al (2010) False-positive findings at contrast-enhanced breast MRI: a BI-RADS descriptor study. AJR Am J Roentgenol 194:1658–1663

    Article  PubMed  Google Scholar 

  25. Baltzer PAT, Kaiser WA and Dietzel M (2015) Lesion type and reader experience affect the diagnostic accuracy of breast MRI: a multiple reader ROC study. Eur J Radiol 84(1):86–91. doi:10.1016/j.ejrad.2014.10.023.

  26. Jansen SA, Shimauchi A, Zak L et al (2011) The diverse pathology and kinetics of mass, nonmass, and focus enhancement on MR imaging of the breast. J Magn Reson Imaging 33:1382–1389

    Article  PubMed  PubMed Central  Google Scholar 

  27. Dietzel M, Baltzer PAT, Schön K, Kaiser WA (2012) MR-mammography: high sensitivity but low specificity? New thoughts and fresh data on an old mantra. Eur J Radiol 81:S30–S32

    Article  PubMed  Google Scholar 

  28. Thomassin-Naggara I, Trop I, Chopier J et al (2011) Nonmasslike enhancement at breast MR imaging: the added value of mammography and US for lesion categorization. Radiology 261:69–79

    Article  PubMed  Google Scholar 

  29. Pinker K, Bickel H, Helbich TH et al (2013) Combined contrast-enhanced magnetic resonance and diffusion-weighted imaging reading adapted to the “Breast Imaging Reporting and Data System” for multiparametric 3-T imaging of breast lesions. Eur Radiol 23:1791–1802

    Article  CAS  PubMed  Google Scholar 

  30. Baltzer A, Dietzel M, Kaiser CG, Baltzer PA (2015) Combined reading of contrast enhanced and diffusion weighted magnetic resonance imaging by using a simple sum score. Eur Radiol. doi:10.1007/s00330-015-3886-x

    PubMed Central  Google Scholar 

Download references

Acknowledgments

The scientific guarantor of this publication is Pascal A. T. Baltzer. The authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article. The authors state that no funding was received for this work. One of the authors has significant statistical expertise. Institutional Review Board approval was obtained. Written informed consent was waived by the Institutional Review Board. Methodology: retrospective, diagnostic or prognostic study, performed at one institution.

Author information

Authors and Affiliations

  1. Department of Biomedical Imaging and Image-guided Therapy, Division of Molecular and Gender Imaging, Medical University of Vienna, Vienna General Hospital, Floor 7F Waehringer Guertel 18-20, 1090, Vienna, Austria

    Maria Adele Marino, Paola Clauser, Ramona Woitek, Georg J. Wengert, Panagiotis Kapetas, Maria Bernathova, Katja Pinker-Domenig, Thomas H. Helbich & Pascal A. T. Baltzer

  2. Department of Biomedical Sciences and Morphologic and Functional Imaging, Policlinico Universitario G. Martino, University of Messina, Messina, Italy

    Maria Adele Marino

  3. Department of Medical and Biological Sciences, Institute of Diagnostic Radiology, Azienda Ospedaliero-Universitaria, “S. Maria della Misericordia”, P.le Santa Maria della Misericordia, University of Udine, Udine, Italy

    Paola Clauser

  4. Diagnosezentrum Meidling, Vienna, Austria

    Klaus Preidler

Authors
  1. Maria Adele Marino
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paola Clauser
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ramona Woitek
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Georg J. Wengert
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Panagiotis Kapetas
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Maria Bernathova
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Katja Pinker-Domenig
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Thomas H. Helbich
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Klaus Preidler
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Pascal A. T. Baltzer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pascal A. T. Baltzer.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Marino, M.A., Clauser, P., Woitek, R. et al. A simple scoring system for breast MRI interpretation: does it compensate for reader experience?. Eur Radiol 26, 2529–2537 (2016). https://doi.org/10.1007/s00330-015-4075-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00330-015-4075-7

Keywords

Search

Navigation