{
  "abstract": "Background Automated classification of gastrointestinal (GI) findings during endoscopy could reduce diagnostic variability and support less experienced endoscopists. While convolutional neural networks (CNNs) have been the standard approach, vision transformer architectures have demonstrated superior performance in general image classification. However, their effectiveness across the full spectrum of GI findings, including rare but clinically significant pathologies, remains insufficiently evaluated.We benchmarked vision transformer architectures against CNN baselines for 23-class GI disease classification, with specific focus on per-class performance and the impact of class imbalance on rare pathological findings.Methods Four pretrained architectures: ResNet50, EfficientNet-B0, ViT-B/16, and Swin Transformer-Tiny, were fine-tuned on HyperKvasir (10,662 images, 23 classes; 8,529/1,063/1,070 train/val/test). Class imbalance spanned a 191× ratio (1,148 to 6 images). Weighted random sampling and class-weighted cross-entropy loss were applied to mitigate imbalance. Models were trained for 20 epochs using AdamW (lr=1e-4) with cosine annealing on an NVIDIA A100 GPU at 224×224 resolution. Metrics included top-1/top-3 accuracy and macro- and weighted-F1.Results Swin Transformer-Tiny achieved the highest top-1 accuracy (89.0%; top-3: 97.5%; weighted F1: 0.888), followed by ViT-B/16 (87.0%), EfficientNet-B0 (86.0%), and ResNet50 (66.7%). Transformers consistently outperformed CNNs, with Swin-T demonstrating the fastest convergence.However, macro-averaged F1 (0.587) was substantially lower than weighted F1 (0.888), exposing a critical failure on rare classes. All 6 classes with fewer than 50 training samples, including Barrett’s oesophagus, hemorrhoids, and early-grade ulcerative colitis variants, achieved F1 = 0.000 across all architectures. A sharp threshold emerged: classes with >100 training samples consistently achieved F1 >0.94, while those below 50 uniformly failed. The performance gap between architectures was modest (~3 percentage points), whereas the gap between common and rare classes within the same model was catastrophic (F1: 0.96 vs 0.00). Class imbalance, not architecture selection, is the dominant bottleneck.Conclusions Swin Transformer achieves state-of-the-art GI classification and is recommended over CNNs. However, no architecture overcame severe class imbalance for rare findings, precisely where AI assistance is most clinically valuable. Future work must prioritise synthetic data augmentation, few-shot learning, and hierarchical classification. Deploying multi-class GI classifiers without addressing rare class performance risks, false reassurance on the findings that matter most.",
  "authors": [
    {
      "affiliations": [
        "New York Medical College, United States"
      ],
      "name": "Jeril Lasington"
    },
    {
      "affiliations": [
        "Rutgers University, United States"
      ],
      "name": "Lawin Steve Mathew Lasington"
    }
  ],
  "title": "IDDF2026-ABS-0160 Can deep learning architectures reliably classify the full spectrum of gastrointestinal endoscopic findings? a comparative study of vision transformers and CNNs on 23-class classification",
  "uid": "33c4ac85-ca0b-52c9-9e4b-8a83d5c56135"
}