Introduction

Contrast-enhanced computed tomography (CT) plays a central role in staging of pancreatic cancer [1]. The staging system depends on the tumour size and location, extension beyond the confinement of the pancreas, adjacent vessels contact or encasement/occlusion, and the presence of distant metastatic disease (e.g. lungs, liver, bones, lymph nodes, and peritoneum). The presence of distant metastatic disease excludes patients from curative resection intent. In addition, based on which vessel is involved and the degree of involvement, patients are excluded or included from curative resection intent. In two recently published reviews [1, 2], one comparing CT with EUS showed lower diagnostic values for CT in evaluating vascular invasion (sensitivity 63% and 72% and specificity 92% and 89%) and the other review summarizing CT findings showed high sensitivity and specificity of 77% and 81%, respectively. Although the sensitivity value in the review by Yang et al is low, CT remains the most commonly used modality, because of the evaluation of distant metastases in a one-stop-shop strategy [3–6].

However, there are concerns that in patients with (borderline) resectable tumours on CT, a substantial number of patients (40%) are found to be unresectable during surgical exploration, based on the presence of either vascular invasion or metastatic disease [7]. This means that unnecessary surgical exploration/laparotomy is performed in 40% of these patients. In a recent review the additional role of laparoscopy before surgical exploration can reduce this percentage to 20% [7]. Several studies also showed the significant role of additional PET-CT to reduce the number of unnecessary surgical explorations [8–11]. However, both PET-CT and laparoscopy are operationally high cost and invasive methods. Therefore, CT remains playing a major initial role in the overall staging of pancreatic cancer

The most important outcome in determining resectability by CT is the positive predictive value for resectability, as in general the patients with (borderline) resectable tumours on CT will undergo surgical exploration followed by resection.

In patients found to be unresectable on CT it is not ethical to perform an invasive technique, and in addition abovementioned reviews [1, 2] showed that CT has a low number of patients with false positive vascular invasion. As this is an important feature in determining unresectablity, the number of patients falsely judged to be unresectable by CT is very low with high predictive values for unresectablity between 89-100% [3]. Therefore, summarizing the predictive values for unresectablity seems not to be relevant.

The aim of this study is to obtain a summary positive predictive value of contrast-enhanced CT in determining the resectability and to report the incidence of contra-indicative factors in the false positive patients (patients judged to be resectable on CT while on reference standard these patients were found to be unresectable).

Materials and methods

Search strategy

Computerized searches in MEDLINE and EMBASE databases from JAN 2005 to JUN 2015 were performed to identify relevant abstracts. For MEDLINE the following keywords were used: "Pancreatic Neoplasms"[Mesh] ”AND ("Tomography, X-Ray Computed"[Mesh] OR "Multidetector Computed Tomography"[Mesh] OR "Four-Dimensional Computed Tomography"[Mesh] OR "Spiral Cone-Beam Computed Tomography"[Mesh] OR "Cone-Beam Computed Tomography"[Mesh] OR "Tomography Scanners, X-Ray Computed"[Mesh] OR "Tomography, Emission-Computed"[Mesh] OR "Tomography, Spiral Computed"[Mesh] OR "Tomography, Emission-Computed, Single-Photon"[Mesh]).

For EMBASE the following text words were used: pancreatic cancer (tw) AND computed tomography (tw) or *computer assisted tomography (tw).

Selection of eligible articles

All retrieved hits were evaluated by a reviewer (X1), with experience in data-extraction of 20 meta-analyses on diagnostic accuracy studies. All titles and abstracts were screened in four steps.

  1. Step 1:

    duplicate papers, letters/comments/editorial/conference abstracts, not pancreatic cancer relevant papers and papers describing other type of pancreatic cancer were excluded.

  2. Step 2:

    case reports, studies on animals/cell lines/phantoms/children, reviews/guidelines, studies only evaluating treatment, studies evaluating prognosis/survival (not imaging related), studies clearly evaluating other imaging (biopsy, FNA), and studies reported in Chinese, Korean, Japanese, and Russian were subsequently excluded.

  3. Step 3:

    subsequently studies were excluded if other parameters based on CT were evaluated; CT was only used for screening or response monitoring or recurrent/follow-up or prognosis/association.

  4. Step 4:

    To identify additional studies, reference lists of relevant articles were checked manually and additional search was performed between JUN 2015 and DEC 2015.

Inclusion and exclusion criteria

All potentially eligible articles were double-checked on inclusion and exclusion criteria by the same reviewer with a delay of 4 weeks. Inclusion criteria were: (a) more than 25 patients with pancreatic adenocarcinoma, (b) patients undergoing contrast-enhanced CT, (c) contrast-enhanced CT for determining resectability, (d) histopathology (surgery, biopsy, or cytology), laparoscopy/laparoscopic ultrasound, follow-up, or consensus were used as reference test, (e) absolute numbers of true positive and false positive results available or could be extracted. Exclusion criteria were: (a) data on same outcome of same study population (study with the largest population was included) and (b) results on combination of different imaging modalities presented and cannot be differentiated for contrast-enhanced CT.

Data extraction

Of the included papers, data were extracted by two reviewers independently (X1 and X2, radiologist with experience in MRI imaging of the pancreas) and discrepancies were resolved by consensus. The following data (including quality assessment) were extracted:

Study design

Year of publication, study period, country of origin, setting (single-centre/multicentre), department of first author, type of data-collection (prospective/retrospective/unclear), and whether ethical approval was obtained (yes/no/unclear).

Patient characteristics

Patient population (suspected/diagnosed/underwent surgery), inclusion/exclusion criteria, number of patients with pancreatic carcinoma included/analysed, number of patients included in total in study, distribution of total number of patients (carcinoma, other malignant lesions, benign lesions), age of patients (mean + SD or median + range), sex ratio (male: female), number of patients undergoing surgery/resection.

Quality criterion was whether consecutive/random sample of patients were enrolled (yes/no/unclear) [12].

Imaging techniques

Bowel preparation, intravenous contrast agent (type, concentration, and amount), phased used (scan delay and reconstructed slice width if available).

Quality criterion was whether the execution of CT was described in sufficient detail to permits its replication (yes/no/unclear) [12]. The execution of CT was described in sufficient detail if type of scanner and type, amount and concentration of contrast agent and phases with scan delay were described

Image evaluation

Method of reconstructions (e.g. multiplanar reconstruction), observers (number, experience, and data on interobserver analysis), and CT criteria used for resectability/unresectablity. Quality criteria were: 1) whether the interpretation of CT was described in sufficient detail to permit its replication (yes/no/unclear). The interpretation of CT was described in sufficient detail if method of reconstruction, number/experience of observers, and criteria for resectability/unresectablity were given; 2) whether CT results were interpreted without knowledge of the results of the reference standard (yes/no/unclear); and 3) if resectability/unresectability criteria were pre-specified (yes/no/unclear) [12].

Reference standard

Data on composition of the reference standard (surgery, resection/histopathology, biopsy/aspiration, laparoscopy with or without ultrasound, follow- up, consensus) were extracted. Quality criterion was whether the reference standard was likely to correctly classify the target condition (yes/no/unclear). In case of consensus including CT, the reference standard was assessed as not correctly classifying the target condition.

Time interval between CT and reference standard

The time interval between CT and reference standard was recorded. Quality criterion was whether there was an appropriate interval [<1 month for surgery, resection/histopathology, biopsy/aspiration, laparoscopy with or without ultrasound, and < 12 months for follow-up between CT and reference standard (yes/no/unclear)] [12].

Data on resectability/unresectability

We extracted data by using CT resectability categories as were reported in the articles versus the two reference standard categories. The data was, therefore, expressed as follows:

  1. a)

    2 × 2 table (resectable OR unresectable on CT vs. resectable OR unresectable on references standard).

  2. b)

    1 × 2 table (resectable on CT vs. resectable OR unresectable on references standard).

True positive is defined as resectable on both CT and reference standard.

False positives are defined as resectable on CT, while unresectable on reference standard.

Of all false positive resectable tumours (judged to be resectable on CT while unresectable on reference standard), the contraindicative factors were recorded if available.

Data-analysis

Publication bias

To study publication bias, we constructed funnel plot for the positive predictive value (PPV) and the Egger regression test was used to examine funnel plot asymmetry [13]. PPV was defined as TP (resectable on both CT and reference standard)/TP + FP (total number of resectable patients on CT). We placed the PPV on the x-axis and the sample size on the y-axis. A p-value of < 0.05 was considered as showing significant publication bias.

Summary positive predictive value

The I 2 statistic, including 95% confidence intervals (CI), was used for quantification of heterogeneity of the PPV [14, 15]. We used either nonlinear fixed-effects (I 2 ≤ 25%) or random-effects (I 2 > 25%) approach to obtain summary PPV. Mean logit PPV with corresponding standard errors were obtained, and then antilogit transformation was performed to calculate summary estimates of PPV (sPPV) including 95% confidence intervals (95%CI) [16, 17].

Exploratory analysis

To study effect of several quality items on the summary PPV (sPPV), we incorporated these items in the model: (1) publication year; (2) design (multicentre vs. single-centre); (3) department of first authors (radiology vs. other); (4) data collection (prospective vs. retrospective/unclear); (5) patient selection (consecutive vs. not consecutive/unclear); (6) detailed description of CT techniques (yes vs. no/unclear); (7) detailed description of CT interpretation (yes vs. no/unclear); (8) blinded interpretation of CT (yes vs. no/unclear); (9) CT criteria described (yes vs. no/unclear); (10) appropriate interval between CT and reference standard (yes vs. no/unclear); and (11) reference standard correctly classify target condition (yes vs. no/unclear). Year of publication was explored as continuous factor, and all other factors were explored as binomial. We considered factors to be explanatory if the corresponding regression coefficients were significantly different from zero, meaning that p-values were less than 0.05.

Subgroup analysis

The following subgroups were defined a priori:

  1. 1.

    Studies using bolus triggering versus studies using fixed timing.

  2. 2.

    Studies including both pancreatic and portal phases versus studies including only portal phase.

  3. 3.

    Studies taking all criteria (liver metastases, other distant metastases, lymph node involvement, local advanced and vascular invasion) into account as criteria for unresectablity versus studies taking only vascular invasion as criteria for unresectablity.

The z test was performed to analyse differences in logit PPV estimates between subgroups. All data analyses were performed by using software (Microsoft Excel 2000, Microsoft, Redmond, Wash; SPSS 10.0 for Windows, SSPS, Chicago, IL, USA; SAS 9.3, SAS Institute, Cary, NC, USA).

Results

Search strategy and selection of eligible articles

The search strategy resulted in 3496 articles. After excluding all non-relevant papers, 125 articles were found to be potentially relevant. An additional 16 articles were found after manually cross-checking, resulting in 141 relevant articles. These articles were checked on inclusion and exclusion criteria. Finally, 29 [18–46] fulfilled the inclusion criteria and data-extraction was performed. The search strategy and selection of eligible articles are shown in Supplement 1 and Fig. 1. The excluded articles are presented in Supplement 1.

Fig. 1
figure 1

Search, selection and inclusion of relevant papers. *Not relevant (other disease, pancreatitis, lung cancer, ovarian cancer, lipoma, neuroendocrine, insulinoma, melanoma, colon carcinoma, renal cell carcinoma, myeloma, colorectal cancer, pancreatitis, endocrine, intracranial, colitis, hernia, polyposis.). †Other type of pancreatic cancer: pseudopapillary, cystadenoma, acinar. ‡Evaluation of other parameters using CT (e.g. qualitative analysis, tumour volume, interobserver). §Case-control studies: evaluation of techniques in patients with pancreatic cancer vs. control (healthy or pancreatitis or other type of tumour, predefined). II Potentially relevant studies: evaluation of CT in patients with suspected, diagnosed pancreatic cancer

Full size image

Study design

Most studies were performed in multicentre setting, initiated by the department of radiology and retrospectively performed. In case a study was prospectively designed [22, 30, 37, 41], resectability by CT was mostly retrospectively analysed. All data on study design characteristics are shown in Table 1.

Table 1 Study design characteristics of the included studies
Full size table

Patient characteristics

In most studies, patients with known pancreatic adenocarcinoma were included. In total, 2171 patients were included (fulfilling inclusion and exclusion criteria) or formed the study group, with ages ranging from 15 to 84 years with a pooled mean of 63.5 years. In 25 studies the distribution of male:female ratio was given as 1088:800.

In some studies [19, 28, 30, 36, 40, 41, 43, 44] data on other malignancies were also given; however, as most of the patients had adenocarcinoma, we did not exclude these studies. All patient characteristics per study are given in Table 2.

Table 2 Patient characteristics of the included studies
Full size table

CT technical features

The number of detectors ranged from 1 to 64. In several studies the specification of CT [18, 35, 43] was not reported. In most of studies, no information on oral contrast administration was given. Intravenous contrast was clearly given in most studies ranging from 90-200 mL. The timing of scanning was either fixed (mostly old studies) or bolus triggering. Pancreatic (late arterial, early portal) and portal (late portal) phases were performed in almost all studies, except in four studies [19, 22, 23, 44]. In ten studies execution of the CT was not described in sufficient detail due to missing information on type of scanner, the type/amount/concentration of iv contrast, and the different phases with scan delay [18, 20, 22, 25, 31, 34, 35, 37, 43, 44]. All details on CT technical features per study are given in Table 3.

Table 3 CT technical features of the included studies
Full size table

Interpretation of CT

Methods of reconstruction and the experience of observers were poorly described. However, the criteria used for unresectability was described in most of the papers, except in three studies [30, 35, 45]. In three studies [24, 25, 41], only vascular invasion was used as criteria for unresectability; in other studies all criteria were used. The interpretation of CT was not described in sufficient detail due to missing information on description of reconstruction methods, number and experience of observers and the criteria for resectability/unresectablity. Whether CT interpretation was blinded was not clear in most of the studies. All data in details are given in Table 4.

Table 4 Interpretation of CT in included studies
Full size table

Reference standard and time interval between CT and reference standard

The interval time between CT and reference standard were only mentioned in ten studies [21, 26–28, 32, 36, 38, 42, 45, 46]. Most reference concerned surgery followed by resection. All details on reference standard and time interval between CT and reference standard are given in Supplement 2.

Data on resectability

As most studies were retrospectively performed, data on 2 x 2 tables or 1 x 2 tables were available, for details see Table 5.

Table 5 Data on resectability by CT
Full size table

Publication bias

The regression coefficient showed no significant relationship between sample size and PPV. The coefficient was 0.47 (95%CI: -0.06-0.99) with a p-value of 0.08 (see also Fig. 2).

Fig. 2
figure 2

Funnel plot showing no relationship between sample size and PPV

Full size image

Summary Positive Predictive Value (sPPV)

The I2 value was 68% (95%CI: 57-76%), indicating that data were heterogeneous. The sPPV was: 81% (95%CI: 75-86%). The false positives were mostly liver metastases, peritoneal metastases, or lymph node metastases. This means that these metastases were missed at CT (Table 7).

Exploratory analysis

Only description of CT technical features, description of interpretation of CT and blinded interpretation of CT had effect on summary PPV. All other items did not have an effect (all p > 0.05).

Studies where CT was described in details compared to studies where CT was not described in details: 85% (95%CI: 79-89%) and 72% (95%CI: 61-81%) respectively (p = 0.013).

Studied where CT interpretation was described in details compared to studies where CT interpretation was not described in details: 89% (95%CI: 83-93%) and 74% (95%CI: 67-80%), respectively (p = 0.004).

Studies where CT interpretation was blinded compared to studies where CT interpretation was not clear: 87% (95%CI: 81-91%) and 72% (95%CI: 63-80%), respectively (p = 0.001).

Subgroup analyses

Studies using bolus-triggering for CT scanning had a slightly higher sPPV compared to studies using fixed timing, respectively 87% (95%CI: 81-91%) and 78% (95%CI 66-86%); however, with a p-value of 0.077.

No significant difference was observed between studies including both pancreatic and portal phases vs. studies including only portal phase, respectively 84% (95%CI:78-88%) and 75% (95%CI: 60-85%), p = 0.153.

No difference was seen between studies taking all criteria into account compared to studies taking only vascular invasion as criteria for unresectablity, respectively 81% (95%CI: 76-86%) and 81% (95%CI: 45-96%), p = 0.984.

Discussion

This meta-analysis showed a sPPV of 81% (95%CI: 75-86%) for predicting resectability by CT. This means that the percentage of patients falsely undergoing surgical exploration is 19%. The false positives (resectable on CT while unresectable on reference standard) for resectability were mostly distant metastases such as liver metastases, peritoneal metastases, or lymph node metastases.

Surprisingly, when checking the subgroup analyses, no significant difference was found between studies including both pancreatic and portal phases (sPPV: 84%) vs. studies including only portal phase (sPPV: 75%). One would except that if both phases are combined, the number of false positive would be significantly reduced, as both phases has a complementary role in the staging of pancreatic cancer [3–6]

The pancreatic phase is the most important phase for detecting and staging a pancreatic tumour. In this phase there is optimal attenuation difference between the hypodense tumour and the normal enhancing pancreatic parenchyma. This phase is not only adequate in detecting primary tumour, but also in delineating periarterial tumour spread in relation to the celiac artery, celiac artery branches, and mesenteric arteries and veins as well (local staging).

The subsequent portal phase has a scan-delay of 70-80 s. At that moment the normal liver parenchyma will enhance optimally, because normal liver cells get 80% of their blood supply through the portal venous system. Liver metastases do not get their blood supply from the portal venous system and will be seen in this phase as hypodense lesions. This phase is, therefore, accurate for detection of liver lesions and is also used for the overall assessment of the abdomen to look for lymph nodes and peritoneal metastases.

We found no significant difference between both phases and only portal phase. This might be explained by the low number of studies using only portal phase, the heterogeneity within data or because the portal phase is also helpful for local staging of the tumour (primary goal of the pancreatic phase).

Based on the higher PPV, frequently used, and the complementary role of both phases, the use of both phases should be continued for evaluation of diagnosis, local staging (vascular invasion), and evaluation of metastatic disease.

Also, no differences were seen between studies taking all criteria (vascular invasion, liver metastases, peritoneal metastases, or lymph node metastases) into account compared to studies taking only vascular invasion as criteria for unresectablity. We expected less false positives when using all criteria, but the sPPV estimates were comparable 81% and 81%.

Most false positives for resectability in both subgroups were due to the presence of distant metastases such as liver metastases, peritoneal metastases, or lymph node metastases. This means that r distant metastases are missed by radiologists even if they are paying attention. In none of the studies, however, was the interpretation of liver or peritoneal metastases defined, and in general it is known that the role of imaging in detecting metastatic lymph nodes is still disappointing. This has also been shown in a recent published systematic review on the diagnostic accuracy of CT in assessing extra-regional lymph node metastases in pancreatic and peri-ampullary cancer, with a mean summary sensitivity of 25% [47].

Even though the distant metastases are missed even if paying attention, these features should be taken into account in the interpretation of CT for determining resectability and thereby defining criteria for especially the peritoneal and lymph node metastases, in order to further reduce the number of unnecessary surgical explorations.

Studies using bolus triggering had a slightly higher sPPV compared to studies using fixed timing, respectively, 87% and 78%; however, with a p-value of 0.077. In general study design and methodological criteria were poorly described. Most of them did not have effect on the outcome, except the description of CT technical features, description of CT interpretation and blinded interpretation of CT. For all these three items, it seems that in case the criteria were clearly described the sPPV raised. The role of these items also has been studied by different methodological group and was found to be relevant and, therefore, STARD 1 and STARD 2 have been developed on optimal reporting diagnostic accuracy studies [48, 49].

So far different reviews have been published on the diagnostic value of CT in evaluating vascular invasion [1, 2] and focusing on vascular invasion with promising results. Only one review published in 2005 evaluated the role of CT in determining resectability, however, they reported summary sensitivity and specificity [50] and not on the positive predictive value of CT. Several studies have been published on this topic after 2005.

We included all studies, hence also studies including a minor portion of patient with other cancer than adenocarcinoma. But most of the patients included were patients with adenocarcinoma. In addition, although the fixed scanning is an older method, there are still centres using this technique and, therefore, we also included these studies.

Most studies were retrospectively performed, and therefore, we were also able to include data on the patients with unresectable tumours on CT. But we did not summarize the negative predictive value (predictive value for unresectability), as it is known that these predictive values are high and in daily practice only patients with (potentially) resectable tumours on CT will undergo reference standard such as surgical exploration, and therefore, no data on reference standard by surgical exploration will be available in a prospective settings. Only one study was performed prospectively and all patients were verified by surgery/follow-up [19].

Summary positive predictive value (sPPV) of CT for determining resectability is high, however, still missing a significant number of patients with distant metastases such as liver metastases, peritoneal metastases, or lymph node metastases. In the paper of Allen et al [7], the false positive could be reduced from 40% to 17% using laparoscopy. Comparable findings are found in our meta-analysis when using only CT. However, a false positive rate of 19% is still high. In several studies the value of PET-CT has been studies in the staging of pancreatic cancer [8–11, 43, 51, 52] and showed to have an additional role in the staging, and thereby even reducing the number of unnecessary surgical exploration [11, 43, 51, 52].

Recommendation is to perform an additional imaging in the patients with (potentially) resectable pancreatic tumours to reduce the number of unnecessary surgical exploration.