Colorectal cancer is the third most common cancer in men and the second in women. Accurate loco-regional staging, primarily based on magnetic resonance imaging (MRI) and rectal endoscopic ultrasound (R-EUS), is essential for treatment planning and relies on assessing tumor depth (T) and regional lymph nodes (N) according to the TNM system. In situ (Tis) and submucosal (T1) tumors may be suitable for minimally invasive endoscopic resection, whereas up to one-third of T2 tumors present mesorectal lymph node metastases, requiring MRI for more accurate staging. Only a few studies have explored artificial intelligence, particularly deep learning (DL), for EUS-based detection and staging of rectal tumors.

This prospective, single-center pilot study evaluated the diagnostic performance of a DL-based tool for detecting and staging rectal tumors on R-EUS images. The model used a convolutional neural network (CNN) with a ResNet backbone pretrained on ImageNet for image classification and segmentation. Performance was assessed using accuracy, precision, recall, F1-score, and spatial agreement metrics between model and expert segmentations (Dice similarity coefficient, DSC). The model was tested for its ability to distinguish Tis–T1 from T2–T3 tumors, and separately, Tis from all other stages.

Fifty patients were prospectively enrolled from May 2023 to December 2024. Tumor segmentation showed a median DSC of 0.65 (IQR 0.17). Pathological tissue detection achieved an accuracy of 0.77 (precision 0.85, recall 0.77, F1-score 0.81). For classifying Tis–T1 vs. T2–T3, accuracy was 0.64 (precision 0.88, recall 0.64, F1-score 0.74). For distinguishing Tis from other stages, accuracy reached 0.80 (precision 0.83, recall 0.89, F1-score 0.86). Mesorectal lymph node segmentation yielded a median DSC of 0.62 (IQR 0.17).

The DL-based tool may assist less-experienced operators in identifying candidates for minimally invasive endoscopic resection. The semi-supervised training produced diagnostic performance comparable to expert assessment, supporting the feasibility of DL-enhanced R-EUS for loco-regional staging of rectal cancer.