Precise intraprocedural phase recognition during complex endoscopic procedures like endoscopic submucosal dissection (ESD) could facilitate automatic objective reporting, specific target-oriented training interventions and measurable quality control. Artificial intelligence algorithms have already been applied to phase recognition in laparoscopic operations and peroral endoscopic myotomy successfully. The aim of this study was to develop and validate an algorithm for automated intraprocedural phase recognition during ESD on a single frame basis.   

The ESD procedure was divided into 5 macro phases: diagnostics, marking, needle injection, dissection and bleeding. Macro-phases were subdivided into micro-phases, which included  scope manipulation, electric current application and injection, leading to a total of 11 phases. A training dataset was compiled from 92 full-length ESD videos (7.930.412 frames) and each frame allocated to one phase. All procedures were performed with the same endoscope system (GIF EZ1500, Olympus, Tokyo, Japan) A video swin transformer was trained in the recognition of the procedural phases. Temporal information was incorporated by uniform frame sampling from past and future of the analyzed frame. The algorithm was validated internally on a validation set (16 ESD procedures, 1.580.394 frames). External validation was performed on 8 ESD procedures (759.346 frames) from a live pig study using a different endoscopy system (GIF H190, Olympus, Tokyo, Japan). In a further evaluation, 2 videos from the external validation set were incorporated into the training data. 

The overall internal validation yielded an accuracy of 86% and an F1 score of 86%. The external validation showed values of 69% and 70% for the same parameters. The evaluation for macro phases showed overall accuracies of 89% and 80% and F1 scores of 90% and 80% for internal and external test sets, respectively. Incorporation of 2 videos from the test set into the training set led to the following results: The accuracy and F1-score for all phase validation were 78% and 78%. For macro-phase evaluation, these values were 87% and 87%. 

Automated procedural phase recognition yielded precise overall performance in internal validation. External validation with different image quality, endoscopy series and subject showed lower, but still adequate performance. Incorporation of a very limited number of videos from the test set into the training improved the performance. Adaptation of a core algorithm to local settings by limited retraining may mitigate problems of generalizability in the future. Further research should determine the value of this technology for automatic reporting and quality control.