Aims
Atrophy extends beyond the cardia is open atrophy and closely related to gastric cancer. However, there is difficulty in determining cardia atrophy under endoscopy.
Methods
This is multicenter prospective study, a total of 895 patients were assigned to the training and validation sets to establish a cardia atrophy diagnostic model. 220 patients were included in the internal test set. Additionally, a total of 354 patients were included in the external test set. In the internal and external test sets, 3 senior and 3 junior endoscopists made online diagnoses, comparing the performance differences to the AI-assisted system.
Results
In the internal test dataset, the accuracy, sensitivity, and specificity of the AI-assisted cardia atrophy diagnostic system were 0.927, 0.941, and 0.919, respectively, with a PPV of 0.878, NPV of 0.962, and AUC of 0.967. In the external test dataset, the accuracy, sensitivity, and specificity were 0.918, 0.894, and 0.931 respectively, with a PPV of 0.874, NPV of 0.943, and AUC of 0.953. Furtherly, it was found the diagnosis accuracy, sensitivity, specificity, PPV, and NPV of the AI-assisted system were all higher than those of senior endoscopists. The ROC curve area (AUC) of AI-assisted system was 0.967 and 0.953 in the internal and external test sets, respectively. Heat map showed a high consistency between the AI-system and endoscopists.
Comparison of diagnostic performance of cardia atrophy between AI-assisted system and endoscopists
|
|
Internal Testing dateset |
External Testing dateset |
||||||
|
|
AI-assisted system |
Endos-copists |
Senior |
Junior |
AI-assisted system |
Endos-copists |
Senior |
Junior |
|
Accuracy (95%CI) |
0.927 (0.891, 0.964) |
0.714 (0.688, 0.741) |
0.774 (0.738, 0.810) |
0.655 (0.617, 0.692) |
0.918 (0.881, 0.956) |
0.836 (0.817, 0.856) |
0.867 (0.843, 0.891) |
0.805 (0.777, 0.833) |
|
Sensitivity (95%CI) |
0.941 (0.884, 0.997) |
0.687 (0.642, 0.731) |
0.730 (0.673, 0.787) |
0.643 (0.582, 0.704) |
0.894 (0.827, 0.961) |
0.770 (0.734, 0.806) |
0.841 (0.795, 0.887) |
0.699 (0.651, 0.747) |
|
Specificity (95%CI) |
0.919 (0.876, 0.963) |
0.732 (0.697, 0.766) |
0.802 (0.757, 0.846) |
0.662 (0.613, 0.710) |
0.931 (0.894, 0.969) |
0.872 (0.849, 0.894) |
0.882 (0.851, 0.913) |
0.862 (0.830, 0.894) |
|
PPV (95%CI) |
0.878 (0.820, 0.936) |
0.612 (0.569, 0.656) |
0.694 (0.636, 0.753) |
0.540 (0.483, 0.597) |
0.874 (0.807, 0.941) |
0.762 (0.727, 0.796) |
0.791 (0.750, 0.832) |
0.729 (0.683, 0.775) |
|
NPV (95%CI) |
0.962 (0.930, 0.993) |
0.791 (0.759, 0.823) |
0.828 (0.786, 0.869) |
0.750 (0.707, 0.794) |
0.943 (0.905, 0.981) |
0.877 (0.855, 0.899) |
0.912 (0.886, 0.938) |
0.843 (0.812, 0.874) |
|
AUC (95%CI) |
0.967 (0.945-0.988) |
- |
- |
- |
0.953 (0.929-0.977) |
- |
- |
- |
PPV: Positive Predictive Value NPV: Negative Predictive Value AUC: Area Under Curve
Conclusions
Our newly developed AI-assisted system shows superior performance in identifying cardia atrophy compared to senior endoscopists, and can be used to guide endoscopic judgment and targeted biopsies for cardia atrophy.