Seminar article
Statistical consideration for clinical biomarker research in bladder cancer

https://doi.org/10.1016/j.urolonc.2010.02.011Get rights and content

Abstract

Objective

To critically review and illustrate current methodological and statistical considerations for bladder cancer biomarker discovery and evaluation.

Methods

Original, review, and methodological articles, and editorials were reviewed and summarized.

Results

Biomarkers may be useful at multiple stages of bladder cancer management: early detection, diagnosis, staging, prognosis, and treatment; however, few novel biomarkers are currently used in clinical practice. The reasons for this disjunction are many and reflect the long and difficult pathway from candidate biomarker discovery to clinical assay, and the lack of coherent and comprehensive processes (pipelines) for biomarker development. Conceptually, the development of new biomarkers should be a process that is similar to therapeutic drug evaluation—a highly regulated process with carefully regulated phases from discovery to human applications. In a further effort to address the pervasive problem of inadequacies in the design, analysis, and reporting of biomarker prognostic studies, a set of reporting recommendations are discussed. For example, biomarkers should provide unique information that adds to known clinical and pathologic information. Conventional multivariable analyses are not sufficient to demonstrate improved prediction of outcomes. Predictive models, including or excluding any new putative biomarker, need to show clinically significant improvement of performance in order to claim any real benefit. Towards this end, proper model building, avoidance of overfitting, and external validation are crucial. In addition, it is important to choose appropriate performance measures dependent on outcome and prediction type and to avoid the use of cutpoints. Biomarkers providing a continuous score provide potentially more useful information than cutpoints since risk fits a continuum model. Combination of complementary and independent biomarkers is likely to better capture the biological potential of a tumor than any single biomarker. Finally, methods that incorporate clinical consequences such as decision curve analysis are crucial to the evaluation of biomarkers.

Conclusions

Attention to sound design and statistical practice should be delivered as early as possible and will help maximize the promise of biomarkers for patient care. Studies should include a measure of predictive accuracy and clinical decision-analysis. External validation using data from an independent cohort provides the strongest evidence that a model is valid. There is a need for adequately assessed clinical biomarkers in bladder cancer.

Introduction

Biomarker research is an all encompassing term for investigation of biologic processes and their potential use in early detection, diagnosis, monitoring of disease, and treatment decision. Most biomarkers represent changes in proteins or genes associated with disease that can be measured. Since there are innumerable alterations associated with disease states, it is necessary to develop statistical tools to identify which changes are significant and which can be clinically useful.

The recent mapping of the human genome, together with advances in high-throughput genetic and proteomic technology, have led to an explosion of biological information and the identification of a plethora of candidate molecular biomarkers and therapeutic targets. Furthermore, work in proteomics is also improving our understanding of systems biology. As the proteome changes constantly with the state of the organism, biological variations that occur over time can easily be addressed. Proteomics therefore has the potential expected to discover new oncologic biomarkers that could help diagnose cancer at an early stage or establish tumor-specific profiles that could predict tumor aggressiveness [1]. High-throughput platforms for analysis of protein expression and functionality in clinical samples and protein microarrays have been developed. These array platforms provide a quantitative or semiquantitative means for measuring protein expression and also provide the ability to identify post-translational modifications, such as phosphorylation. Their increasing use in health research will significantly help the rapid validation and therefore translation of the massive amounts of data generated by modern genomic and proteomic technologies, thereby fostering a new synergy between bench and bedside. This explosion of knowledge about the basic biological processes and the genetics of cancer has led to increasing optimism that this knowledge can be put to practical clinical use in the near future for individualized medicine [2], [3]. Indeed, important examples of translational approaches can already be found in the areas of drug discovery and development, disease diagnosis and classification, selection of therapeutic regimens for individual patients, and designing clinical trials. These are important developments but, as with any new approach, there is a danger of unwarranted enthusiasm and premature clinical application of laboratory results based on insufficient evidence. To carry out the translation of knowledge into practice with maximal efficiency and effectiveness, it is essential to conduct studies with appropriate designs and analyses based on sound statistical principles.

There is considerable evidence that contemporary biomarker research falls far short of using appropriate research design and statistical methods. For example, Vickers et al. reviewed a ‘snapshot’ sample of 129 studies to determine whether the statistical analyses used would allow conclusions to be drawn regarding the clinical value of the biomarkers studied. The authors found that the majority of articles regarding biomarkers for cancer focused on testing the null hypothesis of no association between the biomarker and cancer outcome with only a small minority (11%) reporting predictive accuracy. Not a single paper used a statistical method or study design that addressed whether use of the marker in practice would improve clinical outcome. Other authors have reported on different statistical problems in marker research, such as the correct application of statistical tests, multiple hypothesis testing, data-dependent choice of cutpoints, power, and missing data [4], [5], [6]. Finally, deficiencies in study design, representative at-risk population, definition of endpoints, standard prognostic factor data, or laboratory methods may also limit the conclusions of a study [7]. Given these pervasive problems with study design and data analysis and reporting, this article attempts to address some of the statistical considerations needed for appropriate biomarker discovery and evaluation with focus on bladder cancer.

Section snippets

What exactly is a biomarker? Refining the definition

According to the National Institutes of Health (NIH), a biomarker is a characteristic that is objectively measured and evaluated as an indicator of normal biological processes, pathogenic processes or pharmaceutical responses to a therapeutic intervention [8], [9]. The term biomarker is now typically shorthand for a molecular biomarker. There is a wide range of variation in the complexity of biomarkers going from the simplest (hair color, blood pressure, or cholesterol levels) to more

Biomarker conundrum in bladder cancer

A PubMed Search on “bladder cancer” AND (“biomarker” OR “molecular marker”) in English language yielded 3,434 hits (accessed 12/25/09; Fig. 1). The number of articles published on bladder cancer biomarkers have increased steadily since 1980s, reaching close to 300 articles per year in 2009. Despite this plethora of biomarkers reported to be clinically “promising,” only one biomarker is routinely used by urologists—cytology [13]. Why are bladder cancer biomarkers not living up to their promise

The discovery-validation-implementation paradigm: A schema for biomarker development and evaluation

Conceptually, the development of new biomarkers should be a process that is similar to therapeutic drug evaluation. Drug development is a highly regulated process with carefully regulated phases from discovery to human applications. In 2002, the National Cancer Institute's Early Detection Research Network developed a five phase approach to systematic discovery and evaluation of biomarkers [30], [31]. The phases of research are generally ordered according to the strength of evidence that each

Reporting of biomarker data

In an effort to address the pervasive problem of inadequacies in the design, analysis, and reporting of biomarker prognostic studies, a set of reporting recommendations, such as the REMARK, has been developed and adopted by many prominent journals (Fig. 3). The goal of these guidelines is to encourage transparent and complete reporting and to help readers judge the data and understand the context in which the conclusions apply. Indeed, it provides a detailed description as to the minimum amount

Statistical consideration during early phases of biomarker discovery and validation

“Most people use statistics like a drunk uses a streetlight, for support instead of illumination,” Andrew Lang.

An issue that has received less attention is the degree to which research on biomarkers has made sufficient use of clinically relevant statistics, such as the assessment of predictive accuracy, decision-analysis, and/or experimental methodology. The fundamental idea behind the concept of personalized medicine is that it is possible to identify patterns of demographic, clinical,

Cutpoints for biomarker classification

Many biomarkers are measured on a continuous scale such as, for example, nuclear matrix protein 22 (NMP22), which varies from 0.1 ng/mL to over 100 ng/mL. A common inclination with continuous markers is to chose a cutpoint to distinguish positive from negative results, such as with 10 ng/mL for NMP22 [53].

There are numerous reasons to be suspicious of this practice. First, use of cutpoints makes no biologic sense, on the grounds that it is implausible that there exists a fixed threshold of a

Predictive tools

Traditionally, physician judgment has formed the basis for risk estimation, patient counseling, and decision making. However, clinicians' estimates are often biased due to both subjective and objective confounders [54], [55], [56], [57]. To obviate this problem and to obtain more accurate predictions, researchers have developed predictive tools that are based on statistical techniques [58]. Recently, predictive tools have been introduced to predict the outcome of interest for the individual

Perspectives

“Statistics can be made to prove anything—even the truth,” Author unknown.

Biomarkers have the potential to be used clinically to screen for, diagnose, or monitor the activity of diseases and to guide molecular targeted therapy or assess therapeutic response (Table 1). However, discovery experiments have often overemphasized the significance of novel biomarkers, and efforts to further credential such candidates have been rare. Open any journal today and you will find multiple articles on the

References (73)

  • M.D. Winget et al.

    Development of common data elements: The experience of and recommendations from the early detection research network

    Int J Med Inform

    (2003)
  • V.B. Lokeshwar et al.

    Bladder tumor markers beyond cytology: International Consensus Panel on bladder tumor markers

    Urology

    (2005)
  • S.F. Shariat et al.

    Perspectives on prostate cancer biomarkers

    Eur Urol

    (2008)
  • E.W. Steyerberg et al.

    Internal validation of predictive models: Efficiency of some procedures for logistic regression analysis

    J Clin Epidemiol

    (2001)
  • E.W. Steyerberg et al.

    Internal and external validation of predictive models: A simulation study of bias and precision in small samples

    J Clin Epidemiol

    (2003)
  • Y. Lotan

    Role of biomarkers to predict outcomes and response to therapy

    Urol Oncol

    (2010)
  • S.F. Shariat et al.

    Risk stratification for bladder tumor recurrence, stage, and grade by urinary nuclear matrix protein 22 and cytology

    Eur Urol

    (2004)
  • P.L. Ross et al.

    A catalog of prostate cancer nomograms

    J Urol

    (2001)
  • S.F. Shariat et al.

    Nomograms for bladder cancer

    Eur Urol

    (2008)
  • S.P. Lerner et al.

    The use and abuse of data: Nomograms and talking to patients about clinical medicine

    Urol Oncol

    (2007)
  • A.C. Wolff et al.

    American Society of Clinical Oncology/College of American Pathologists guideline recommendations for human epidermal growth factor receptor 2 testing in breast cancer

    J Clin Oncol

    (2007)
  • S.F. Shariat et al.

    Critical review of prostate cancer predictive tools

    Future Oncol

    (2009)
  • A.J. Vickers et al.

    Cutpoints in clinical chemistry: Time for fundamental reassessment

    Clin Chem

    (2009)
  • E.W. Steyerberg et al.

    Assessing the performance of prediction models: A framework for traditional and novel measures

    Epidemiology

    (2010)
  • B.J. Schmitz-Drager et al.

    Bladder cancer markers in patient management: The current perspective

    World J Urol

    (2008)
  • Biomarkers and surrogate endpoints: Preferred definitions and conceptual framework

    Clin Pharmacol Ther

    (2001)
  • R.S. Svatek et al.

    Screening for bladder cancer using urine-based tumor markers

    Minerva Urol Nefrol

    (2008)
  • H.B. Grossman et al.

    Detection of bladder cancer using a point-of-care proteomic assay

    JAMA

    (2005)
  • M.P. Herman et al.

    Urine-based biomarkers for the early detection and surveillance of non-muscle invasive bladder cancer

    Minerva Urol Nefrol

    (2008)
  • S.J. Schnitt

    Traditional and newer pathologic factors

    J Natl Cancer Inst Monogr

    (2001)
  • S.F. Shariat et al.

    Improved prediction of disease relapse after radical prostatectomy through a panel of preoperative blood-based biomarkers

    Clin Cancer Res

    (2008)
  • S.F. Shariat et al.

    p53, p21, pRB, and p16 expression predict clinical outcome in cystectomy with bladder cancer

    J Clin Oncol

    (2004)
  • S.F. Shariat et al.

    Multiple biomarkers improve prediction of bladder cancer recurrence and mortality in patients undergoing cystectomy

    Cancer

    (2008)
  • F. Andre et al.

    Molecular classification of breast cancer: Implications for selection of adjuvant chemotherapy

    Nat Clin Pract Oncol

    (2006)
  • E. Check

    Proteomics and cancer: Running before we can walk?

    Nature

    (2004)
  • E.P. Diamandis

    Proteomic patterns to identify ovarian cancer: 3 Years on

    Expert Rev Mol Diagn

    (2004)
  • Cited by (0)

    View full text