Check for updates
Development and validation of prognostic nomograms in patients with adrenocortical carcinoma: a population-based study
Hao Zhang1,2 . Yaser Naji1 . Minbo Yan1 . Wenfei Lian1 . Maochun Xie1 . Yingbo Dai1,2D
Received: 8 January 2020 / Accepted: 12 February 2020 Springer Nature B.V. 2020
Abstract
Background Predicting the prognosis of patients with adrenocortical carcinoma (ACC) is difficult, due to its unpredictable behavior. The aim of this study is to develop and validate a nomogram to predict survival outcomes in patients with ACC. Methods Nomograms were established using the data collected from the Surveillance, Epidemiology, and End Results (SEER) database. Based on univariate and multivariate Cox regression analyses, we identified independent risk factors for overall survival (OS) and cancer-specific survival (CSS). Concordance indexes (c-indexes), the area under the receiver operating characteristics curve (AUC) and calibration curve were used to evaluate predictive performance of these models. The clinical use of nomogram was measured by decision curve analysis (DCA) and clinical impact curves.
Results A total of 855 eligible patients, randomly divided into training (n =600) and validation cohorts (n=255), were included in this study. Based on the independent predictors, the nomograms were established and demonstrated good discriminative abilities, with C-indexes for OS and CSS were 0.762 and 0.765 in training cohorts and 0.738 and 0.758 in validation cohorts, respectively. The AUC and calibration plots also demonstrated a good performance for both nomograms. DCA indicated that the two nomograms provide clinical net benefits.
Conclusion We unveiled the prognostic factors of ACC and developed novel nomograms that predict OS and CSS more accurately and comprehensively, which can help clinicians improve individual treatment, making proper clinical decisions and adjusting follow-up management strategies.
Keywords Adrenocortical carcinoma · Nomogram · Decision curve analysis . Surveillance . Epidemiology · End results
Introduction
Adrenocortical carcinoma (ACC) is a rare and highly aggres- sive solid tumor with a poor prognosis. It is reported that the incidence is approximately 0.72 per million cases per year and leading to 0.2% of all cancer deaths in the United States [1]. ACC Characterized by high recurrence rate and the low ☒ rate of response to postsurgical therapies, making it the sec- ond most aggressive endocrine malignancy [2-4]. Although most cases are treated surgically, the median survival only
☒ Yingbo Dai daiyb@mail.sysu.edu.cn
1 Department of Urology, The Fifth Affiliated Hospital, Sun Yat-Sen University, No. 52, Meihua East Road, Zhuhai 519000, Guangdong, China
2 Guangdong Provincial Key Laboratory of Biomedical Imaging, The Fifth Affiliated Hospital, Sun Yat-Sen University, Zhuhai, Guangdong, China
ranges between 14 and 28 months [5]. However, the prog- nosis of ACC is highly unpredictable, making it a challeng- ing disease to treat. Some patients survive for more than 10 years despite metastatic disease, whereas others die within a few months from a rapidly progressive disease not responding to any available therapy [6]. Therefore, estab- lishing a clinical prognostic model is essential for improv- ing individual treatment, making proper clinical decisions and adjusting follow-up management strategies. Due to the scarcity of cases, to the best of our knowledge, there are few reports on the clinical prognostic model of ACC. Zini et al. [7] built a model for predicting mortality of ACC with three variables (age, stage and surgical status) but it lacks spe- cific indicators of prediction ability. Li et al. [8] constructed nomograms that predict overall survival (OS) and cancer- specific survival (CSS) of ACC by logistics regression, and the C-index (an evaluation index) was 0.677 for the model predicting OS and 0.672 for CSS. In view of the existing individual prediction model of ACC is not satisfactory and
precise enough, we tried to build a more accurate and com- prehensive prognostic model in this study.
Materials and methods
Patients and study design
The data of patients diagnosed as ACC between 1975 and 2016 were retrieved from the Surveillance, Epidemiol- ogy, and End Results (SEER) 21 registry database using SEER*Stat 8.3.6 software. SEER database collects clinical information on various cancer types from population-based cancer registries covering approximately 34.6% of the US population [9]. The flow chart of the data selection process is shown in Fig. 1. The cohort for this analysis consisted of patients diagnosed with ACC between 1975 and 2016. The histological types of ACC were limited using the site code 74.0/74.9 and the International Classification of Diseases for Oncology-3(ICD-O-3): 8370. The inclusion criteria were as follows: (1) patients with positive histology, (2) patients with complete dates and there are more than 0 days of survival, (3) patients with ACC as the primary tumor or is the first one of multiple tumors. The exclusion criteria were as follows:
(1) patients with autopsy/death certificate only, (2) patients with incomplete tumor information (tumor size, extension, lymph nodes and metastasis). Finally, a total of 855 cases in the SEER cohort were included and analysed in this study. All ACC patients were randomly assigned at a ratio of 7:3 to either the training set for nomograms or the validation set for validation. The cutoff value of age at diagnosis and tumor size were calculated by X-tile 3.6.1 software (Yale University, New Haven, Connecticut, USA) which was ini- tially developed to determine the best cutoff values for vari- ables in breast malignancy [10]. The optimal cutoff values for age were identified as 44 and 58 years while the best cutoff values for tumor size were 85 mm (Fig. 2). TNM stage of patients was defined according to the UICC/AJCC TNM staging system [11]. The clinicopathological features which were used in our research included age at diagnosis, gender, ethnicity, marital status, laterality, tumor size, tumor grade, T stage, N stage, M stage, use of radiotherapy, use of chemo- therapy and use of surgery.
Development of the nomograms
We used univariate and multivariable Cox regression analy- sis to screen out risk factors and independent prognostic
Patients diagnosed as ACC between 1975 and 2016(primary site code: 74.0 / 74.9,ICD-O-3 histologic type code:8370) (N=2182)
I
Patients with positive histology (N=2066)
Excluded patients with autopsy/death certificate only (N=2031)
I
Patients with complete dates and there are more than 0 days of survival (N=1760) Patients with one primary only or ACC is the first of 2 or more primaries (N=1532)
I
Excluded patients with incomplete tumor information ( tumor size, extension, lymph nodes and metastasis ) (N=855)
/
Training cohort (n=600)
Validation cohort (n=255)
A
B
86
100-
No. of Patients
% Survival
50
0
0.0
44.0
58.0
90.0
0
Age(years)
0.0
5.0
10.0
Survival Time(years)
C
D
99-
1007
No. of Patients
% Survival
50
0
5.0
85.0
385.0
0
Size(mm)
0.0
Survival Time(years)
5.0
10.0
factors for OS and CSS in training cohort, respectively. Hazard ratios and corresponding 95% confidence intervals (95% CI) of variables were also calculated. All variables were screened using the forward stepwise selection method in the multivariate Cox regression model [12, 13]. Nomo- grams for 1-, 3- and 5-years OS and 1-, 3- and 5-years CSS were constructed on the basis of the identified independent prognostic factors.
Validation of the nomograms
To ensure the accuracy of our nomograms, we validated the nomograms both internally and externally. The consist- ency index (C-index) and the area under the time-dependent receiver operating characteristic curve (AUC) were used to
evaluate the predictive performance of each nomogram. Fur- thermore, we compared the nomograms and the TNM stage using the C-index. Then, we applied a bootstrapping method with 1000 resamples to build Calibration plots which can evaluate the agreement between the actual probability and predicted probabilities from the constructed nomograms [14].
Clinical performance
Decision curve analysis (DCA) was performed to identify and compare the clinical application value between the nom- ogram model and other clinical features by calculating the net benefits at each risk threshold probability [15, 16]. The net benefit was decided by subtracting the proportion of all
false-positive results from the proportion of true-positive results and weighted by the relative harm caused by abne- gating treatment compared with the negative consequences of unnecessary treatments [17]. According to the principle of DCA, we evaluated the clinical impact of the nomogram in comparison with TNM stage to help us more intuitively understand its significant value by plotting curves.
Statistical analysis
All statistical analyses were performed using the software SPSS version 25.0 (IBM, Armonk, NY, USA) and R soft- ware version 3.6.1 (Vienna, Austria; https://www.R-proje ct.org) for Windows. OS was defined as the time from diag- nosis to the time of death from any cause or the most recent follow-up, and CSS was calculated from the date of diag- nosis to the date of ACC-related death or the most recent follow-up. Continuous variables such as age and tumor size were converted into categorical variables in the light of the result of X-tile. All the categorical variables were presented with frequencies and proportions, and compared between the training and validation cohorts by the chi-square test for validating that the clinicopathological features between the two groups were well balanced. All tests were two sided, and P <0.05 was considered indicative of statistical significance.
Ethical statement
We had signed the Data-Use Agreement for the SEER 1975-2016 Research Database File and for SEER Radia- tion Therapy and Chemotherapy Information. This study was deemed exempt by the Ethics Committee and Institutional Review Board of The Fifth Affiliated Hospital of Sun Yat- sen University, for it is based on the data extracted from the publicly available SEER database.
Results
Patient baseline characteristics
In total, 855 eligible ACC patients diagnosed from 1975 to 2016 in the SEER database were identified and randomly assigned into a training set (n = 600) and a validation set (n =255). The demographic of patients and clinical charac- teristics of the tumor in the training and validation cohorts are summarized in Table 1, which shows no significant differences between the two cohorts (P>0.05). A total of 274 (32.0%) patients were under 44 years old, 265(31.0%) patients were between 45 and 58, and 316 (37.0%) patients were age 59 and above. The majority of patients in the pre- sent study were women (60.8%) and of white race (82.8%). More than half of the patients (57.2%) were married.
Among them, 46.4% of the patients had tumors on the right, 51.9% had on the left, and 1.6% had bilateral tumors or unknown. In 284 patients (33.2%), the tumor size was less than or equal to 85 mm; while 571 (66.8%) of patients had a tumor size of more than 85 mm. Of the pathologic grades of tumor, 53 (6.2%) patients were Grade I or Grade II, 131 (15.3%) patients were Grade III or Grade IV, and 671 (78.5%) patients were not available. Most patients (61.6%) were diagnosed as T3 or T4, and only a few patients had regional lymph node invasion (14.4%) and distant metastasis (33.3%) at diagnosis. Moreover, surgery was performed in 712 patients (83.3%), while chemotherapy and radiotherapy were performed in only a small part of patients [121 (14.2%) and 377(44.1%), respectively].
Independent significant factors in the training set
According to the univariate analysis, OS was significantly associated with age, tumor size, grade, TNM stage, chemo- therapy, and surgery. Whereas no significant differences were observed in OS concerning gender, ethnicity, marital status, laterality and radiotherapy (Table 2). Significant fac- tors identified by univariate analysis were further explored in multivariate analysis; which demonstrated that age (P<0.05), tumor grade, TNM stage, and surgery, with a P value of <0.01 each, were the independent prognostic fac- tors (Fig. 3). Similar results were observed in CSS (Table 2, Fig. 3).
Development of nomograms for OS and CSS
Based on the independent prognostic factors identified in the multivariate Cox regression analysis, we developed two nomograms to predict 1-, 3-, and 5-year OS (Fig. 4a) and CSS (Fig. 4b) in ACC patients. The nomogram for OS prediction indicated that surgery contributed most to the outcome, followed by age, M stage, and tumor grade. As for CSS, the nomogram revealed that surgery was also the most crucial factor affecting prognosis, followed by M stage, tumor grade and age.
Validation of nomograms for OS and CSS
The C-indexes in the nomograms and TNM stage in both cohorts are listed in Table 3. The C-index for the OS pre- diction nomogram was 0.762 (95% CI 0.738-0.786) for the training group and 0.738 (95% CI 0.699-0.777) for the validation group, which was higher than the correspond- ing C-indexes for the TNM stage. Furthermore, the C-index for CSS prediction nomogram in the training and validation groups was 0.765 (95% CI 0.740-0.790) and 0.758 (95% CI 0.717-0.799), respectively, which was also higher than the TNM stage. As can be seen from Fig. 5a-d, the AUC
| Variables | Training cohort (n=600) | Validation cohort (n=255) | Total (n=855) | P value |
|---|---|---|---|---|
| Age (years) | 0.414 | |||
| ≤ 44 | 184 (30.7) | 90 (35.3) | 274 (32.0) | |
| 45~58 | 190 (31.7) | 75 (29.4) | 265 (31.0) | |
| ≥ 59 | 226 (37.7) | 90 (35.3) | 316 (37.0) | |
| Gender | 0.549 | |||
| Male | 239 (39.8) | 96 (37.6) | 335 (39.2) | |
| Female | 361 (60.2) | 159 (62.4) | 520 (60.8) | |
| Ethnicity | 0.737 | |||
| Black | 53 (8.8) | 23 (9.0) | 76 (8.9) | |
| White | 500 (83.3) | 208 (81.6) | 708 (82.8) | |
| Other/unknown | 47 (7.8) | 24 (9.4) | 71 (8.3) | |
| Marital status | 0.361 | |||
| Married | 345 (57.5) | 144 (56.5) | 489 (57.2) | |
| Single | 236 (39.3) | 107 (42.0) | 343 (40.1) | |
| Unknown | 19 (3.2) | 4 (1.6) | 23 (2.7) | |
| Laterality | 0.638 | |||
| Right | 284 (47.3) | 113 (44.3) | 397 (46.4) | |
| Left | 307 (51.2) | 137 (53.7) | 444 (51.9) | |
| Bilateral/unknown | 9 (1.5) | 5 (2.0) | 14 (1.6) | |
| Tumor size (mm) | 0.557 | |||
| ≤ 85 | 203 (33.8) | 81 (31.8) | 284 (33.2) | |
| > 85 | 397 (66.2) | 174 (68.2) | 571 (66.8) | |
| Tumor grade | 0.152 | |||
| Grade I/II | 36 (6.0) | 17 (6.7) | 53 (6.2) | |
| Grade III/ IV | 83 (13.8) | 48 (18.8) | 131 (15.3) | |
| Unknown | 481 (80.2) | 190 (74.5) | 671 (78.5) | |
| T stage | 0.625 | |||
| T1/T2 | 227 (37.8) | 101 (39.6) | 328 (38.4) | |
| T3/T4 | 373 (62.2) | 154 (60.4) | 527 (61.6) | |
| N stage | 0.064 | |||
| N0 | 505 (84.2) | 227 (89.0) | 732 (85.6) | |
| N1 | 95 (15.8) | 28 (11.0) | 123 (14.4) | |
| M stage | 0.081 | |||
| M0 | 411 (68.5) | 159 (62.4) | 570 (66.7) | |
| M1 | 189 (31.5) | 96 (37.6) | 285 (33.3) | |
| Radiotherapy | 0.401 | |||
| No/unknown | 519 (86.5) | 215 (84.3) | 734 (85.8) | |
| Yes | 81 (13.5) | 40 (15.7) | 121 (14.2) | |
| Chemotherapy | 0.713 | |||
| No/unknown | 333 (55.5) | 145 (56.9) | 478 (55.9) | |
| Yes | 267 (44.5) | 110 (43.1) | 377 (44.1) | |
| Surgery | 0.284 | |||
| No | 95 (15.8) | 48 (18.8) | 143 (16.7) | |
| Yes | 505 (84.2) | 207 (81.2) | 712 (83.3) |
Values are expressed as n (%)
values are proven to be sensitive in predicting the 1-, 3-, and 5-year OS. Both the C-indexes and AUC values suggested that these models made accurate predictions and had good
discriminative abilities. The calibration curves for the prob- ability of 3- and 5-year OS demonstrated good consistency between the nomogram prediction and actual survival in the
| Variables | Cancer-specific survival | Overall survival | ||||
|---|---|---|---|---|---|---|
| HR | 95% CI | P value | HR | 95% CI | P value | |
| Age (years) | ||||||
| ≤ 44 | Reference | Reference | ||||
| 45~58 | 1.333 | 1.008-1.763 | 0.044 | 1.502 | 1.149-1.963 | 0.003 |
| ≥ 59 | 1.992 | 1.532-2.590 | <0.001 | 2.322 | 1.805-2.988 | < 0.001 |
| Gender | ||||||
| Male | Reference | Reference | ||||
| Female | 1.066 | 0.859-1.323 | 0.56 | 1.113 | 0.910-1.361 | 0.299 |
| Ethnicity | ||||||
| Black | Reference | Reference | ||||
| White | 1.010 | 0.578-1.766 | 0.972 | 1.051 | 0.626-1.763 | 0.852 |
| Other/unknown | 1.081 | 0.733-1.593 | 0.694 | 1.069 | 0.744-1.537 | 0.719 |
| Marital status | ||||||
| Married | Reference | Reference | ||||
| Single | 0.8026 | 0.642-1.004 | 0.0544 | 0.8428 | 0.684-1.038 | 0.107 |
| Unknown | 1.2059 | 0.674-2.158 | 0.528 | 1.049 | 0.586-1.878 | 0.871 |
| Laterality | ||||||
| Right | Reference | Reference | ||||
| Left | 0.718 | 0.318-1.622 | 0.426 | 0.6587 | 0.323-1.343 | 0.251 |
| Bilateral/unknown | 0.714 | 0.316-1.614 | 0.418 | 0.6271 | 0.307-1.280 | 0.200 |
| Tumor size (mm) | ||||||
| ≤85 | Reference | Reference | ||||
| >85 | 1.413 | 1.119-1.784 | 0.004 | 1.355 | 1.091-1.683 | 0.006 |
| Tumor grade | ||||||
| Grade I/II | Reference | Reference | ||||
| Grade III/IV | 2.538 | 1.351-4.766 | 0.004 | 2.349 | 1.326-4.160 | 0.003 |
| Unknown | 2.382 | 1.335-4.249 | 0.003 | 2.230 | 1.326-3.751 | 0.003 |
| T stage | ||||||
| T1/T2 | Reference | Reference | ||||
| T3/T4 | 2.29 | 1.799-2.915 | < 0.001 | 2.126 | 1.702-2.657 | < 0.001 |
| N stage | ||||||
| N0 | Reference | Reference | ||||
| N1 | 2.510 | 1.921-3.280 | < 0.001 | 2.499 | 1.942-3.215 | < 0.001 |
| M stage | ||||||
| M0 | Reference | Reference | ||||
| M1 | 3.884 | 3.116-4.840 | < 0.001 | 3.587 | 2.913-4.417 | < 0.001 |
| Radiotherapy | ||||||
| No/unknown | Reference | Reference | ||||
| Yes | 1.003 | 0.728-1.382 | 0.984 | 0.940 | 0.691-1.279 | 0.694 |
| Chemotherapy | ||||||
| No/unknown | Reference | Reference | ||||
| Yes | 1.340 | 1.082-1.658 | 0.007 | 1.240 | 1.015-1.515 | 0.036 |
| Surgery | ||||||
| No | Reference | Reference | ||||
| Yes | 0.1443 | 0.110-0.190 | < 0.001 | 0.1506 | 0.116-0.195 | < 0.001 |
CI confidence interval, HR hazard ratio
training cohort (Fig. 6a, b). Similar results were observed for the CSS nomogram (Fig. 6c, d). As for validation set, the
calibration curves for the probability of 3-, 5-year OS and CSS were also in good agreement (Fig. 7a-d).
A
Covariates Age(years)
Hazard Ratio(95%CI) P-value
≤44
Reference
45~58
1.351(1.021-1.788)
0.035
259
2.333(1.780-3.059)
<0.001
Gender
Male
Reference
Female
0.922(0.745-1.140)
0.451
Ethnicity
Black
Reference
White
1.025(0.607-1.729)
0.928
Other/unknown
0.926(0.639-1.342)
0.683
Marital status
Married
Reference
Single
0.926(0.743-1.153)
0.492
Unknown
1.167(0.625-2.181)
0.627
Laterality
Right
Reference
Left
1.127(0.528-2.408)
0.757
Bilateral/unknown
1.124(0.529-2.389)
0.761
Tumor size(mm)
≤85
Reference
>85
1.124(0.895-1.411)
0.314
Tumor Grade
Grade I/II
Reference
Grade III/IV
2.508(1,451-4.335)
<0.001
Unknown
2.386(1,317-4.324)
0.004
T Stage
T1/T2
Reference
Т3/ 4
1.529(1.202-1.944)
<0.001
N Stage
NO
Reference
N1
1.605(1.215-2.120)
<0.001
M Stage
MO
Reference
M1
2.658(2.040-3.463)
<0.001
Radiotherapy
No/Unknown
Reference
Yes
0.826(0.601-1.134)
0.237
Chemotherapy
No/Unknown Yes
Reference
0.802(0.636-1.009)
0.06
Surgery
No
Reference
Yes
.
0.290(0.215-0.392)
<0.001
05
15
The estimates
¥
3.5
B
Covariates
Hazard Ratio(95%CI) P-value
Age(years)
≤44
Reference
45~58
1.161(0.865-1.558)
0.319
≥59
1.926(1.452-2.555)
<0.001
Gender
Male
Reference
Female
0.867(0.690-1.090)
0.221
Ethnicity
Black
Reference
White
1.004(0.571-1.765)
0.988
Other/unknown
0.956(0.643-1.421)
0.825
Marital status
Married
Reference
Single
0.847(0.669-1.073)
0.169
Unknown
1.423(0.756-2.677)
0.274
Laterality
Right
Reference
Left
1.371(0.575-3.269)
0.477
Bilateral/unknown
1.446(0.610-3.430)
0.403
Tumor size(mm)
$85
Reference
>85
1.159(0.908-1.479)
0.236
Tumor Grade
Grade V/II
Reference
Grade III/IV
2.803(1.523-5.159)
<0.001
Unknown
2.745(1.421-5.301)
0.003
T Stage
T1/T2
Reference
T3/T4
1.595(1.230-2.067)
<0.001
N Stage
NO
Reference
N1
1.545(1.151-2.073)
0.004
M Stage
MO
Reference
M1
2.807(2.125-3.706)
<0.001
Radiotherapy
No/Unknown
Reference
Yes
0.874(0.629-1.215)
0.424
Chemotherapy
No/Unknown
Reference
Yes
0.829(0.649-1.059)
0.134
Surgery
No
Reference
Yes
.
0.275(0.200-0.377)
<0.001
05
15
The estimates
35
45
5
15
Clinical performance of nomograms
The DCA results of the nomograms and TNM stage are shown in Fig. 9. It was demonstrated that the nomograms add greater net benefit after 1 and 3 years than that achieved for the other indexes for predicting OS (Fig. 8a, b) and CSS (Fig. 8c, d) in the training set. Similar results were found in the validation cohort (Fig. 8e-h). Furthermore, comparing with TNM stage, the increased net benefit of nomograms indicated that these models were more accurate in predict- ing OS and CSS in patients with ACC in both the training and validation cohorts. Based on the above, we further plot- ted clinical impact curves to evaluate the clinical impact of the nomograms to help us more intuitively realized its significant value. The nomogram for predicting OS dem- onstrated that cost/benefit ratios were lower when the risk threshold was less than 0.6 (Fig. 9a), while the CSS nomo- gram showed that cost/benefit ratios were lower when the risk threshold was less than 0.7 (Fig. 9b) in the training set. Additionally, in the validation set, both the OS nomogram and CSS nomogram revealed that cost/benefit ratios were lower when the risk threshold was less than 0.7 (Fig. 9c, d).
Discussion
ACC is a rare and highly aggressive cancer with a poor prognosis. Due to its unpredictable prognosis, clinical deci- sion-making remains difficult. Therefore, precise individual prognosis prediction and treatment guidelines are urgently needed. Regrettably, the existing individual prediction model of ACC is not satisfactory and precise enough. Hence, this study was carried out to tried to build a more accurate and comprehensive prognostic model.
In this study, we identified several conventional fac- tors including age of diagnosis, tumor size, tumor grade, T stage, N stage, M stage, chemotherapy, and surgery per- formed that significantly affected OS. Out of 855 eligible ACC patients, women (60.8%) and white race (82.8%) had a distinct advantage in quantity, which were consistent with previous research [4, 5, 18]. According to the results of the study, all of gender, ethnicity, marital status, laterality have no significant effect on prognosis. The cutoff value of age at diagnosis and tumor size were measured by X-tile 3.6.1 software. It was shown that the OS and CSS of patients with ACC worsened with increasing age and proved that age is a powerful and independent risk factor for ACC. Although a larger tumor size (> 8.5 cm) resulted in a worse outcome that shown in survival analysis, it was not an independent factor by multivariate Cox regression analysis. Literature is contradicted regarding whether tumor size can directly affect prognosis. While some reported that large tumor size has been related to inferior survival after complete resection [19,
A
0
1
2
3
4
5
6
7
8
9
10
Points
Age(years)
45~58
≤44
≥59
Tumor Grade
Unknown
Grade I/II
T3/T4
Grade III/IV
T Stage
T1/T2
N1
N Stage
NO
M Stage
M1
MO
No
Surgery
Yes
Total Points
0
5
10
15
20
25
30
35
40
45
1-Year OS
0.9
0.8
0.7 0.6 0.50.40.3 0.2 0.1
3-Year OS
0.9
0.8
0.7 0.6 0.50.40.3 0.2 0.1
5-Year OS
0.8
0.7
0.6 0.50.40.3 0.2
0.1
B
0
1
2
3
4
5
6
7
8
9
10
Points
Age(years)
45~58
≤44
≥59
Unknown
Tumor Grade
Grade I/II
Grade III/IV
T Stage
T3/T4
T1/T2
N1
N Stage
NO
M Stage
M1
MO
No
Surgery
Yes
Total Points
0
5
10
15
20
25
30
35
40
45
1-Year CSS
0.9
0.8
0.7
0.6 0.50.40.30.2 0.1
3-Year CSS
0.9
0.8
0.7 0.6 0.50.40.30.2 0.1
5-Year CSS
0.9
0.8
0.7 0.6 0.50.40.30.2 0.1
| Group | Variable | OS CSS | |||
|---|---|---|---|---|---|
| C-index | 95% CI | C-index | 95% CI | ||
| Training cohort | Nomogram | 0.762 | 0.738-0.786 | 0.765 | 0.740-0.790 |
| TNM stage | 0.698 | 0.673-0.723 | 0.711 | 0.686-0.736 | |
| Validation cohort | Nomogram | 0.738 | 0.699-0.777 | 0.758 | 0.717-0.799 |
| TNM stage | 0.694 | 0.651-0.737 | 0.714 | 0.671-0.757 | |
OS overall survival, CSS cancer-specific survival
A
B
1.00
1.00
0.75
0.75
Sensitivity
Sensitivity
0.50
0.50
0.25
0.25-
1-y ROC of OS, AUC=0.846
1-y ROC of CSS,AUC=0.857
3-y ROC of OS, AUC=0.850
3-y ROC of CSS, AUC=0.864
5-y ROC of OS, AUC=0.855
5-y ROC of CSS,AUC=0.864
0.00
0.00
0.00
0.25
0.50
1-Specificity
0.75
1.00
0.00
0.25
0.50
1-Specificity
0.75
1.00
C
D
1.00
1.00
0.75
0.75
Sensitivity
Sensitivity
0.50
0.50
0.25
0.25
1-y ROC of OS, AUC=0.815
1-y ROC of CSS, AUC=0.843
3-y ROC of OS, AUC=0.815
3-y ROC of CSS,AUC=0.829
5-y ROC of OS,AUC=0.831
5-y ROC of CSS,AUC=0.853
0.00
0.00
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
1-Specificity
1-Specificity
20]; others found that it has no significant difference in prog- nosis [1, 21]. In our study, tumor grade was also an inde- pendent prognostic factor for survival. The same conclusions
have been reached in previous reports [1, 5]. Interestingly, the unknown group is still significant when comparing with Grade I/II by Cox regression. The probable reason may be
A
B
0.8
0.8
Actual 3-Year Survival
0.6
Actual 5-Year Survival
0.6
0.4
0.4
0.2
0.2
0.0
0.0
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
n=800 d=390 p=8, 149 subjects per group
Gray: ideal
Nomogram-Predicted Probability of 3-Year OS
X - resampling optimism added, B=1000 Based on observed-predicted
n=600 d=300 p=8, 149 subjects per group Gray: ideal
Nomogram-Predicted Probability of 5-Year OS
X - resampling optimism added, B=1000 Based on observed-predicted
C
D
0.8
0.8
Actual 3-Year Survival
0.6
Actual 5-Year Survival
0.6
0.4
0.4
0.2
0.2
0.0
0.0
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
n=800 d=343 p=8, 149 subjects per group
Gray: ideal
Nomogram-Predicted Probability of 3-Year CSS
X - resampling optimism added, B=1000 Based on observed-predicted
n=600 d=343 p=8, 149 subjects per group
Gray: ideal
Nomogram-Predicted Probability of 5-Year CSS
X - resampling optimism added, B=1000 Based on observed-predicted
that there are a large number of patients with high-grade tumors in the unknown group. Furthermore, Wang et al. [22] revealed that tumor stage had a dominant effect on the cancer outcomes. According to the result, more than half of patients (61.6%) were diagnosed as T3 or T4 stage cancer, which meant the greater part of patients already had local invasion or adjacent organs invasion at the time of diagnosis. As for N and M stages, we indicated that 14.4% of patients had regional lymph node invasion and 33.3% had distant metastasis. The result of the TNM stage was paralleled to the previous study [1, 4, 22, 23].
Complete surgical resection is currently the only known cure for ACC [24]. The majority of patients in this study had undergone operation treatment. Surgery was not only an independent prognostic factor but also contributed most to the outcomes. Unlike surgery, radiotherapy and chemo- therapy just serve as adjuvant or palliative methods. In general, patients receiving chemotherapy were more than those selected for radiation. This might be because ACC has long been thought to be radio-resistant. Although several recent studies revealed the efficacy of radiotherapy in both the adjuvant and palliative settings, it has not been firmly backed up with any prospective data [25, 26]. Strangely,
in the univariate COX regression analysis, we found that patients who received chemotherapy were at higher risk of poorer prognosis than those who did not, which was also found in another study [8]. One explanation for that might be the lack of enough data on patients receiving chemotherapy. And these patients in the unknown group had undergone chemotherapy in practice and enjoyed a long survival. Not- withstanding being lack of the exact mechanism of its antitu- mor action, mitotane is the best studied and most commonly used chemotherapeutic agent for ACC [27]. However, the lack of specific chemotherapeutic drug records in the seer database prevented us from directly studying mitotane or other chemotherapeutic agents.
Recently, nomograms, which possess the ability to visu- ally show data, accuracy, and individualization, are widely used tools for prognosis in oncology and medicine [16]. The simple calculations with straight scales is more acceptable to clinicians. Therefore, we used nomograms to establish mod- els for predicting overall survival and cancer-specific sur- vival in patients with ACC, which revealed reliable discrimi- native and predictive abilities with relatively high C-indexes and values of AUC in both the training and validation set. The calibration plots indicated good agreement between
A
B
0.8
0.8
Actual 3-Year Survival
0.6
Actual 5-Year Survival
0.6
0.4
0.4
0.2
0.2
0.0
0.0
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
n=255 d=152 p=8, 62 subjects per group
Gray: ideal
Nomogram-Predicted Probability of 3-Year OS
X - sesampling optimism added, B=1000 Based on observed-predicted
n=255 d=152 p=8, 62 subjects per group
Gray: ideal
Nomogram-Predicted Probability of 5-Year OS
X - resampling optimism added, B=1000 Based on observed-predicted
C
D
0.8
0.8
Actual 3-Year Survival
0.6
Actual 5-Year Survival
0.6
0.4
0.4
0.2
0.2
0.0
0.0
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
n=255 d=138 p=8, 62 subjects per group
Gray: ideal
Nomogram-Predicted Probability of 3-Year CSS
X - resampling optimism added, B=1000 Based on observed-predicted
n=255 d= 136 p=8, 62 subjects per group
Gray: ideal
Nomogram-Predicted Probability of 5-Year CSS
X- resampling optimism added, B=1000 Based on observed-predicted
predictions and observations in both of the nomograms. By and large, diagnostic performance is evaluated in terms of sensitivity and specificity, but these indicator are not suitable for the clinical utility of specific models [28]. In other words, the established nomograms might not be useful even with a higher value of C-index and perfect calibration. There- fore, using DCA, we evaluated if nomogram-based medical decisions and strategies could improve patient prognoses (equivalent to net benefit) to show the clinical value of the nomograms [17, 29].
Good clinical utility was indicated in a proper range in the present study. Furthermore, clinical impact curves help us more intuitively realized the clinical impact of the nomo- grams. In general, the nomograms for predicting OS and CSS of ACC actually take on good predictive efficiencies as judged by the methods above.
As an example for the application of the nomogram, a 60-year-old patient diagnosed with ACC is discussed. The tumor size is 5.0 cm and there is no extra-adrenal invasion, regional lymph node metastasis and distant metastasis. The patient have undergone surgery and the tumor grade is grade III. Corresponding points can be acquired from the nomo- grams. This patient totally receives 14.8 and 13.5 points in
the OS and CSS prognostic nomograms, respectively. There- fore, the estimated 1-, 3- and 5-year OS probability of this patient would be 77%, 46% and 34%, respectively, from the OS nomogram scale. The 1-, 3- and 5-year CSS probability of this patient would be 82%, 54% and 44%, respectively, from the CSS nomogram scale.
The tumor, lymph node, and metastasis (TNM) classifi- cation system proposed by the International Union Against Cancer (UICC) and the American Joint Committee on Can- cer (AJCC) is the most extensively used classification, which can provide a powerful prognostic tool for predicting both disease-free and disease-specific survival in patients with cancer [30]. Although not accurate enough, TNM staging did provide a reference for predicting the prognosis of ACC. In the present study, through comparing with TNM stage in C-index and DCA, our models showed clear superiority in both predictive abilities and clinical applications.
When comparing with the existing prognostic models of ACC [7, 8], our study also had obvious advantages. First, in our study, the data of patients diagnosed as ACC were derived from SEER database between 1975 and 2016 and 2016 was the year of the latest data for the database. Second, clinicopathological features analyzed in our research were
A
B
TMN Stage
0.6
TMN Stage
Nomograms
Nomograms
0.3
All
All
None
0.5
None
0.4
Net Benefit
0.2
Net Benefit
0.3
0.1
0.2
0.1
0.0
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Threshold Probability
Threshold Probability
C
D
0.3
TMN Stage
1.0
TMN Stage
Nomograms
Nomograms
All
All
None
0.8
None
Net Benefit
0.2
Net Benefit
0.6
0.1
0.4
0.2
0.0
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Threshold Probability
Threshold Probability
E
F
0.25
TMN Stage
1.0
TMN Stage
Nomograms
Nomograms
All
All
0.20
None
0.8
None
Net Benefit
0.15
Net Benefit
0.6
0.10
0.4
0.05
0.00
0.2
-0.05
0.0
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.0
0.2
0.4
0.6
0.8
1.0
Threshold probability
Threshold probability
G
H
1.0
TMN Stage
1.0
TMN Stage
Nomograms
Nomograms
All
All
0.8
None
0.8
None
Net Benefit
0.6
Net Benefit
0.6
0.4
0.4
0.2
0.2
0.0
0.0
0.0
0.1
0.2
0.3
0.4
0.0
0.2
0.4
0.6
0.8
1.0
Threshold probability
Threshold probability
A
B
1000
Number high risk
Number high risk
Number high risk with outcome
1000
Number high risk with outcome
Number high risk (out of 1000)
800
Number high risk (out of 1000)
800
600
600
400
400
200
200
0
O
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
High Risk Threshold
High Risk Threshold
1:100
1:4
2:3
3:2
4:1
100:1
1:100
1:4
2:3
3:2
4:1
100:1
Cost:Benefit Ratio
Cost:Benefit Ratio
C
D
1000
Number high risk
Number high risk with outcome
1000
Number high risk
Number high risk with outcome
Number high risk (out of 1000)
800
Number high risk (out of 1000)
800
600
600
400
400
200
200
0
O
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
High Risk Threshold
High Risk Threshold
1:100
1:4
2:3
3:2
4:1
100:1
1:100
1:4
2:3
3:2
4:1
100:1
Cost:Benefit Ratio
Cost:Benefit Ratio
more comprehensive than previous studies. The variables included age at diagnosis, gender, ethnicity, marital status, laterality, tumor size, tumor grade, T stage, N stage, M stage, use of radiotherapy, use of chemotherapy and use of surgery. Moreover, in terms of the classification of continuous vari- ables (age at diagnosis and tumor size), we exerted X-tile to determine the best cutoff values and converted continuous variables into categorical variables, which may made our models have better discriminative and predictive abilities. Third, the indicators we used to test the predictive power and consistency of these nomograms included C-index, 1-, 3- and 5-year AUC, 3- and 5-year calibration curves, which were more rigorous and complete than others’ study. Fur- thermore, the C-index was 0.762 (95% CI 0.738-0.786) and
0.765 (95% CI 0.740-0.790) for the OS and CSS prediction nomograms in the training cohort, respectively, which were higher than those in the study worked by Li et al. [8]. Finally, to further confirmed that if our models had clinical appli- cation value, we also plotted the DCA and clinical impact curves for our models, which were of great value and had been overlooked in previous articles.
However, potential limitations of the study should not be ignored. First of all, this is a retrospective study that has inherent limitations respect to selection bias. Second, because of the limitations of the seer database, we only identified some of the clinicopathological features to con- struct the nomograms and were not be able to incorporate molecular, genetic markers and hormone, which might
potentially affect predictive accuracy of ACC. For example, high proliferative activity as assessed by Ki-67 staining, evi- dence for mutated p53 and abnormal activation of ß-Catenin are related to the poor prognosis of ACC [19, 31]. Third, recurrence, comorbidities and complications may occur during the follow-up process, which should also be taken into account. Finally, the validation cohort in this study was come from the same SEER dataset as the training set, which may lead to overfitting of the models; thus, external verifi- cation at different databases or multiple centers should be performed.
Conclusion
With clinical and pathological data identified in a large pop- ulation-based cohort, we constructed prognostic nomograms that can objectively and precisely predict OS and CSS of patients with ACC by COX regression. Besides, the training and testing cohort validation results demonstrated that our nomograms showed good performance not only in accuracy but also applicability. These models were confirmed to be clinically useful based on decision curve analysis and can be used as effective and acceptable evaluation tools for cli- nicians to perform individual survival prediction in ACC, potentially resulting in reduced decision-making cost and medical burden.
Acknowledgements We thank the seer database and the data gatherers in every registries.
Compliance with ethical standards
Conflict of interest The authors have declared that no conflict of inter- est exists in this work.
References
1. Kebebew E, Reiff E, Duh QY, Clark OH, McMillan A (2006) Extent of disease at presentation and outcome for adrenocortical carcinoma: Have we made progress? World J Surg 30(5):872-878. https://doi.org/10.1007/s00268-005-0329-x
2. Fassnacht M, Kroiss M, Allolio B (2013) Update in adrenocortical carcinoma. J Clin Endocrinol Metab 98(12):4551-4564. https:// doi.org/10.1210/jc.2013-3020
3. Icard P, Goudet P, Charpenay C et al (2001) Adrenocortical carci- nomas: surgical trends and results of a 253-patient series from the French Association of Endocrine Surgeons Study Group. World J Surg 25(7):891-897. https://doi.org/10.1007/s00268-001-0047-y
4. Ayala-Ramirez M, Jasim S, Feng L et al (2013) Adrenocortical carcinoma: clinical outcomes and prognosis of 330 patients at a tertiary care center. Eur J Endocrinol 169(6):891-899. https://doi. org/10.1530/EJE-13-0519
5. Sharma E, Dahal S, Sharma P et al (2018) The characteris- tics and trends in adrenocortical carcinoma: a United States
Population Based Study. J Clin Med Res 10(8):636-640. https:// doi.org/10.14740/jocmr3503w
6. Allolio B, Hahner S, Weismann D, Fassnacht M (2004) Man- agement of adrenocortical carcinoma. Clin Endocrinol (Oxf) 60(3):273-287. https://doi.org/10.1046/j.1365-2265.2003.01881.x
7. Zini L, Capitanio U, Jeldres C et al (2009) External validation of a nomogram predicting mortality in patients with adrenocortical carcinoma. BJU Int 104(11):1661-1667. https://doi.org/10.1111/ j.1464-410X.2009.08660.x
8. Li Y, Bian X, Ouyang J, Wei S, He M, Luo Z (2018) Nomograms to predict overall survival and cancer-specific survival in patients with adrenocortical carcinoma. Cancer Manag Res 10:6949-6959. https://doi.org/10.2147/CMAR.S187169
9. Cronin KA, Ries LAG, Edwards BK (2014) Preface. Cancer 120(S23):3755-3757. https://doi.org/10.1002/cncr.29049
10. Camp RL, Dolled-Filhart M, Rimm DL (2004) X-tile: a new bio- informatics tool for biomarker assessment and outcome-based cut- point optimization. Clin Cancer Res 10(21):7252-7259. https:// doi.org/10.1158/1078-0432.CCR-04-0713
11. DeLellis RA, Lloyd RV, Heitz PU, Eng C (eds) (2004) Pathology and genetics of tumours of endocrine organs. IARC, Lyon
12. Moons KG, Altman DG, Reitsma JB et al (2015) Transparent reporting of a multivariable prediction model for Individual Prog- nosis Or Diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med 162:W1-W73. https://doi.org/10.7326/M14-0698
13. Sauerbrei W, Boulesteix AL, Binder H (2011) Stability investiga- tions of multivariable regression models derived from low- and high-dimensional data. J Biopharm Stat 21(6):1206-1231. https ://doi.org/10.1080/10543406.2011.629890
14. Royston P, Altman DG (2013) External validation of a Cox prog- nostic model: Principles and methods. BMC Med Res Methodol. https://doi.org/10.1186/1471-2288-13-33
15. Vickers AJ, Cronin AM, Elkin EB, Gonen M (2008) Extensions to decision curve analysis, a novel method for evaluating diagnostic tests, prediction models and molecular markers. BMC Med Inform Decis Mak 8:1-17. https://doi.org/10.1186/1472-6947-8-53
16. Vickers AJ, Elkin EB (2006) Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making 26(6):565-574. https://doi.org/10.1177/0272989X06295361
17. Huang YQ, Liang CH, He L et al (2016) Development and validation of a radiomics nomogram for preoperative predic- tion of lymph node metastasis in colorectal cancer. J Clin Oncol 34(18):2157-2164. https://doi.org/10.1200/JCO.2015.65.9128
18. Bilimoria KY, Shen WT, Elaraj D et al (2008) Adrenocortical carcinoma in the United States: treatment utilization and prognos- tic factors. Cancer 113(11):3130-3136. https://doi.org/10.1002/ cncr.23886
19. Stojadinovic A, Ghossein RA, Hoos A et al (2002) Adrenocor- tical carcinoma: clinical, morphologic, and molecular charac- terization. J Clin Oncol 20(4):941-950. https://doi.org/10.1200/ JCO.2002.20.4.941
20. Harrison LE, Gaudin PB, Brennan MF (1999) Pathologic features of prognostic significance for adrenocortical carcinoma after cura- tive resection. Arch Surg 134(2):181-185. https://doi.org/10.1001/ archsurg. 134.2.181
21. Şişman P, Şahin AB, Peynirci H et al (2017) Adrenokortikal kars- inom: Tek merkez deneyimi. Turkish J Urol 43(4):462-469. https ://doi.org/10.5152/tud.2017.81598
22. Wang S, Chen S-S, Gao W-C et al (2017) Prognostic factors of adrenocortical carcinoma: an analysis of the Surveillance Epi- demiology and End Results (SEER) database. Asian Pac J Can- cer Prev 18(10):2817-2823. https://doi.org/10.22034/APJCP .2017.18.10.2817
23. Paton BL, Novitsky YW, Zerey M et al (2006) Outcomes of adre- nal cortical carcinoma in the United States. Surgery 140(6):914- 920. https://doi.org/10.1016/j.surg.2006.07.035
24. Vaidya A, Nehs M, Kilbridge K (2019) Treatment of adrenocor- tical carcinoma. Surg Pathol Clin 12(4):997-1006. https://doi. org/10.1016/j.path.2019.08.010
25. Sabolch A, Feng M, Griffith K, Hammer G, Doherty G, Ben-Josef E (2011) Adjuvant and definitive radiotherapy for adrenocortical carcinoma. Int J Radiat Oncol Biol Phys 80(5):1477-1484. https ://doi.org/10.1016/j.ijrobp.2010.04.030
26. Fassnacht M, Hahner S, Polat B et al (2006) Efficacy of adjuvant radiotherapy of the tumor bed on local recurrence of adrenocorti- cal carcinoma. J Clin Endocrinol Metab 91(11):4501-4504. https ://doi.org/10.1210/jc.2006-1007
27. Hescot S, Slama A, Lombes A et al (2013) Mitotane alters mito- chondrial respiratory chain activity by inducing cytochrome c oxi- dase defect in human adrenocortical cells. Endocr Relat Cancer 20(3):371-381. https://doi.org/10.1530/ERC-12-0368
28. Fitzgerald M, Saville BR, Lewis RJ (2015) Decision curve analy- sis. JAMA 313(4):409-410. https://doi.org/10.1001/jama.2015.37
29. Kerr KF, Brown MD, Zhu K, Janes H (2016) Assessing the clinical impact of risk prediction models with decision curves: guidance
for correct interpretation and appropriate use. J Clin Oncol 34(21):2534-2540. https://doi.org/10.1200/JCO.2015.65.5654
30. Fassnacht M, Johanssen S, Quinkler M et al (2009) Limited prog- nostic value of the 2004 international union against cancer staging classification for adrenocortical carcinoma. Cancer 115(2):243- 250. https://doi.org/10.1002/cncr.24030
31. Gaujoux S, Grabar S, Fassnacht M et al (2011) B-catenin activa- tion is associated with specific clinical and pathologic charac- teristics and a poor outcome in adrenocortical carcinoma. Clin Cancer Res 17(2):328-336. https://doi.org/10.1158/1078-0432. CCR-10-2006
Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.