The clinicopathological data of 1860 GC patients treated at the Department of Gastric Surgery, Fujian Medical University Union Hospital from January 2011 to April 2015 were analyzed retrospectively. The inclusion criteria were that all patients were diagnosed with GC and accepted radical surgery at our center. The exclusion criteria were as follows: (1) The general clinical data were incomplete, including age, sex, body mass index (BMI), American society of anesthesiologists (ASA) score, carcinoembryonic antigen (CEA), carbohydrate antigen 19-9 (CA199), alpha fetoprotein (AFP), NLR, PLR, AGR, PNI, preoperative complications, tumor size, primary site, clinical T stage (cT), and clinical N stage (cN); (2) The follow-up data were missing; and (3) Patients receiving neoadjuvant chemotherapy. A total of 1608 patients were included (Supplementary Figure 1). The study was approved by the Ethics Committee of Fujian Medical University Union Hospital.

All patients underwent routine preoperative examinations, including upper gastrointestinal endoscopy and upper gastrointestinal angiography, to assess tumor location. Preoperative clinical staging was assessed using chest radiography, total abdominal computed tomography (CT), abdominal ultrasound, and, if necessary, PET-CT, and bone scans. The 8^th edition of the AJCC tumor staging system was used for cTNM and pathological TNM (pTNM) staging (Supplementary Figure 1). The overall survival (OS) time was the time from the operation to the last follow-up time or death.

All patients were randomly divided into a training set (n = 1104, 70%) and a testing set (n = 504, 30%), and the testing set was separated for evaluation of the final model. The training data were used five times for cross-validation to optimize the neural network. Finally, the separated data of the testing set were used to evaluate the model.

Routine blood biochemistry and tumor markers were assessed within 1 wk before the operation. Using X-tile software, the optimal critical values of the NLR for OS were 1.91 and 3.87 (P < 0.05). According to optimal critical values, the patients were divided into three groups as follows: Low NLR group (NLR ≤ 1.91, n = 632), middle NLR group (1.91 < NLR ≤ 3.87, n = 753), and high NLR group (NLR > 3.87, n = 223). Similarly, the patients were divided into a low PLR group (PLR ≤ 89.5, n = 258), middle PLR group (89.5 < PLR ≤ 162.3, n = 777), and high PLR group (PLR > 162.3, n = 573). According to the AGR, the patients were divided into a low AGR group (AGR ≤ 6.59, n = 474), middle AGR group (6.59 < AGR ≤ 7.81, n = 612), and high AGR group (AGR > 7.81, n = 522). Furthermore, according to the PNI, the patients were divided into a low PNI group (PNI ≤ 371.01, n = 495), middle PNI group (371.0 < PNI ≤ 430.0, n = 771), and high PNI group (PNI > 430.0, n = 342).

First, logistic univariate analysis was used to analyze 1104 cases in the training set to screen the variables that affected the 3-year survival and to determine these variable items as the input nodes of the ANN.

We used the multilayer perceptron to develop the preope-ANN (Figure 1). The input layer of the preope-ANN model consisted of the screened general clinicopathological data and preoperative blood biomarkers using logistic analysis and the 3-year survival condition. The ANN intelligently analyzed the 3-year survival of the patients according to the input data and generated the outcome compared with the real value. The output layer consisted of two values: (1) The patient's 3-year survival (death or survival); and (2) The continuous variables from 0 to 1, which represent the predictive probability of the outcome of the patient's 3-year survival. The network architecture of the preope-ANN model was composed of seven hidden layers, and the input layer was composed of 16 nodes (including 14 general clinical data and blood biomarkers and two survival conditions related to the 3-year OS). The hidden layer was activated by the hyperbolic tangent function, and the output layer was composed of two nodes activated by the softmax function, including the survival condition and survival probability. According to the survival probability predicted by the preope-ANN model, the patients were divided into seven subgroups: The survival probability was 0%-30% for group A, 30%-50% for group B, 50%-65% for group C, 65%-80% for group D, 80%-90% for group E, 90%-97% for group F, and 97%-100% for group G.

Open in New Tab   Full Size Figure   Download Figure

Figure 1 Structure diagram of the preoperative artificial neural network model. Age, sex, body mass index, American society of anesthesiologists score, tumor location, tumor size, prognostic nutrition index, albumin-globulin ratio, neutrophil-lymphocyte ratio, platelet-lymphocyte ratio, clinical T stage, clinical N stage, carcinoembryonic antigen, carbohydrate antigen 19-9, and alpha fetoprotein are the input variables. All weighted values passed to the hidden layer node are summed on the hidden layer node and passed to the output node through the sigmoid function. All weighted values entering the output node are summed again and passed through the sigmoid function. For each patient, the probability of output is 0-1.0. In the training of the artificial neural network, the output values are compared with the real results of each patient. The weight is adjusted so that the next time the patient appears on the network, the network output is closer to the real result. NLR: Neutrophil-lymphocyte ratio; PLR: Platelet-lymphocyte ratio; AGR: Albumin-globulin ratio; PNI: Prognostic nutrition index; BMI: Body mass index; ASA: American society of anesthesiologists score; cT: Clinical T stage; cN: Clinical N stage; CEA: Carcinoembryonic antigen; CA199: Carbohydrate antigen 19-9; AFP: Alpha fetoprotein.

All data were analyzed using SPSS 20.0 (SPSS Inc. Chicago, IL, United States) and R software (version 3.5.4). The classified variables were tested using the χ² test or Fisher's exact test. The X-tile software was used to determine the best cut-off point of the counting data. Mann-Whitney U test was used to test the receiver operating characteristic (ROC) curve and the area under the curve (AUC), and the Z test was used to compare the AUCs (MedCal software). We used Harrell’s C index to measure the discriminatory ability of the different models[4,20]. The likelihood ratio chi-square was calculated by Cox regression to measure homogeneity; a higher score means better homogeneity[21]. The Akaike information criterion (AIC) within the Cox regression model was used to compare performances between two prognostic models; smaller AIC values represent better optimistic prognostic stratification[22]. We calculated the relative likelihood of two models using the following formula: Exp {[AIC (model A) - AIC (model B)]/2}. The relative likelihood represents the probability that model A minimizes information as effectively as model B and could thus be interpreted as a P value for the comparison of both AIC values[23]. P < 0.05 was considered statistically significant.

Table 1 shows the clinical data of 1608 patients with GC. Among them, 1104 cases were in the training set, and 504 cases were in the testing set. The average age of all patients diagnosed was 60.72 years (range, 12-101 years), and the male to female ratio was 2.84:1. The proportion of patients diagnosed with stages I, II, III, and IVA disease in the cTNM system was 11.6%, 38.2%, 45.6%, and 4.6%, respectively. The proportion of patients diagnosed with stage I, II, and III disease in the pTNM system was 22.6%, 24.9%, and 52.5%, respectively.

In the training set, the univariate logistics regression analysis (Table 2) showed that age, sex, BMI, CEA, CA199, AFP, NLR, PLR, AGR, PNI, ASA score, tumor location, tumor size, cT stage, and cN stage were significant factors for the 3-year OS of the patients (P < 0.05 for all).

The accuracy, sensitivity, and specificity of the preope-ANN model in the training set were 77.3%, 88.5%, and 50.1%, respectively. For the preope-ANN model in the testing set, the accuracy was 75.2%, the sensitivity was 86.5%, and the specificity was 43.8% (Figure 2).

Open in New Tab   Full Size Figure   Download Figure

Figure 2 Prediction of the 3-year survival of patients using the preoperative artificial neural network. In the training set, the predictive accuracy rate was 77.3%. The sensitivity and specificity of the centralized training model were 88.5% and 50.1%, respectively. The accuracy of the testing set model for survival prediction was 75.2%. The sensitivity and specificity of the test set model were 86.5% and 43.1%, respectively.

In Figure 3, the prediction results of the preope-ANN model were divided into seven subgroups. The Kaplan-Meier survival curve showed that the survival curve of each subgroup within the preope-ANN model had good discrimination in both the training and testing sets (P < 0.05). For cTNM staging, the survival curve in the training set showed (Figure 4) that the substages of the TNM system were well differentiated (P < 0.05). However, the survival curve of the testing set showed that the curve of stage III was close to that of stage IVA (P = 0.335). Figure 5 shows the survival analysis of the pTNM staging. In the training set, the survival curve discrimination between stages IA and IB was poor (P = 0.240), and there was no significant difference between stages IIA and IIB and stages IIB and IIIA (P < 0.05). In the testing set, there was no significant difference in survival curves between stages IA and IB, IIA and IIB, and IIIA and IIIB (P > 0.05).

Open in New Tab   Full Size Figure   Download Figure

Figure 3 Subgroup survival curves of the preoperative artificial neural network. A: The survival curve of each subgroup of the preoperative artificial neural network model showed good discrimination in the training and testing sets (P < 0.05). B: The survival analysis of the training set showed that the substages of the TNM system were well differentiated (P < 0.05). Preope-ANN: Preoperative artificial neural network.

Open in New Tab   Full Size Figure   Download Figure

Figure 4 Survival curves of the clinical TNM staging. A: The discrimination between the survival curves of each stage of the clinical TNM (cTNM) system in the training set was good (P < 0.05). B: There was no significant difference in the survival curves between the cTNM stages I and II A (P = 0.935), and between stages III and IVA (P = 0.355). cTNM: Clinical TNM.

Open in New Tab   Full Size Figure   Download Figure

Figure 5 Survival curves of the pathological TNM staging. A: In the training set, the survival curve discrimination between stages IA and IB was poor (P = 0.240), and there was no significant difference in the survival curves between stages IIA and IIB and IIB and IIIA (P < 0.05). B: In the testing set, there was no significant difference in survival curves between stages IA and IB, IIA and IIB, and IIIA and IIIB of the pathological TNM staging (P > 0.05). pTNM: Pathological TNM.

Figure 6 shows the ROC curves of the preope-ANN model, cTNM stage, and pTNM stage in both the training set and the testing set. In the training set, the AUC values of the preope-ANN model, cTNM stage, and pTNM stage were 0.820 (0.800-0.838), 0.740 (0.718-0.762), and 0.803 (0.782-0.822), respectively. The predictive performance of the preope-ANN was better than that of the cTNM stage (P < 0.05) and was similar to that of the pTNM stage (P = 0.130). In the testing set, the AUC values of the preope-ANN model, cTNM stage, and pTNM stage were 0.790 (0.752- 0.825), 0.687 (0.644-0.727), and 0.786 (0.748-0.821), respectively. Comparison of the AUC showed that the prediction performance of the preope-ANN was better than that of the cTNM stage (P < 0.05), and was similar to that of the pTNM (P = 0.858).

Open in New Tab   Full Size Figure   Download Figure

Figure 6 Comparison of the receiver operating characteristic curves among the preope-ANN model, clinical TNM stage, and pathological TNM stage. A: In the training set, the area under the curve (AUC) values of the preoperative artificial neural network (preope-ANN) model, clinical TNM (cTNM) staging, and pathological TNM (pTNM) staging were 0.820 (0.800-0.838), 0.740 (0.718-0.762), and 0.803 (0.782-0.822), respectively. The comparison of the AUC values in the training set showed that the predictive performance of the preope-ANN was better than that of the cTNM stage (P < 0.05), and similar to that of pTNM stage (P = 0.130). B: In the testing set, the AUC values of the preope-ANN model, cTNM staging, and pTNM staging were 0.790 (0.752-0.825), 0.687 (0.644-0.727), and 0.786 (0.748-0.821), respectively. The comparison of the AUC values in the testing set showed that the prediction performance of the preope-ANN and the pTNM stage was better than that of the cTNM stage (P < 0.05), and the prediction performance of the preope-ANN was similar to that of the pTNM stage (P = 0.858). pTNM: Pathological TNM; cTNM: Clinical TNM; preope-ANN: Preoperative artificial neural network.

As shown in Table 3, for Harrell's C index, the preope-ANN model was superior to the cTNM staging model and had the same performance as the pTNM staging model in both the training and testing sets. (In the training set: Preope-ANN vs cTNM = 0.773 vs 0.663, respectively, P < 0.001; preope-ANN vs pTNM = 0.773 vs 0.757, respectively, P = 0.120; in the testing set: Preope-ANN vs cTNM = 0.752 vs 0.652, respectively, P < 0.001; preope-ANN vs pTNM =0.752 vs 0.740, respectively, P = 0.539). The AIC analysis showed that the preope-ANN model of the training set had a better fitting degree than both the cTNM staging and pTNM staging (preope-ANN vs cTNM = 4977.83 vs 5176.70, respectively, relative likelihood < 0.001; preope-ANN vs pTNM = 4977.83 vs 4999.80, respectively, relative likelihood < 0.001). The fitting degree of the preope-ANN model in the testing set was better than that of the cTNM staging (preope-ANN vs cTNM = 1952.94 vs 2020.37, respectively, relative likelihood < 0.001), and the fitting degree of the preope-ANN model was not inferior to that of the pTNM staging (preope-ANN 1952.94 vs pTNM 1951.84, respectively, relative likelihood = 1.733). Therefore, the performance of the preope-ANN was better than that of the cTNM staging and was similar to that of the pTNM staging.

Because of the powerful abilities of self-learning and handling complex biological information, artificial neural network (ANN) models have been widely applied to disease diagnosis, imaging analysis, and prognosis prediction. However, there has been no trained preoperative ANN (preope-ANN) model to preoperatively predict the prognosis of patients with gastric cancer (GC).

This study combined the preoperative blood biomarkers and preoperative tumor data to establish an ANN model in order to build a reliable preoperative prediction system that can achieve the same effect as postoperative TNM staging. The aim of this study was to evaluate the prognosis of patients with GC and to provide a reasonable individualized treatment plan for patients.

We aimed to establish a neural network model that can predict long-term survival of GC patients before surgery to evaluate the tumor condition before the operation.

The clinicopathological data of 1608 GC patients treated from January 2011 to April 2015 at the Department of Gastric Surgery, Fujian Medical University Union Hospital were analyzed retrospectively. Patients were randomly divided into a training set (70%) for establishing a preope-ANN model and a testing set (30%). The prognostic evaluation ability of the preope-ANN model was compared with that of the American Joint Commission on Cancer (8^th edition) clinical TNM stage (cTNM) and pathological TNM stage (pTNM) through the receiver operating characteristic curve, Akaike information criterion index, Harrell's C index, and likelihood ratio chi-square.

We used the variables that were statistically significant factors for the 3-year overall survival as input-layer variables to develop a preope-ANN in the training set. The survival curves within each score of the preope-ANN had good discrimination (P < 0.05). Comparing the preope-ANN model, cTNM, and pTNM in both the training and testing sets, the preope-ANN model was superior to cTNM in predictive discrimination (C index), predictive homogeneity (likelihood ratio chi-square), and prediction accuracy (area under the curve). The prediction efficiency of the preope-ANN model was similar to that of pTNM.

The preope-ANN model can accurately predict the long-term survival of GC patients, and its predictive efficiency is not inferior to pTNM staging.

This study for the first time confirmed that the preope-ANN is a novel and convenient prognostic model through the use of a large sample data size, which can effectively predict the prognosis of GC patients. In the clinic, preope-ANN can be considered as part of preoperative risk stratification to guide the individualized treatment of patients with GC. The next challenge is to establish a web version of the preope-ANN model that can be dynamically adjusted for the input of different sample data; with this approach, the model accuracy would be closer to the real value and more flexibly applied to the evaluation of clinical patients.