{
  "abstract": "Background Programmed death-ligand 1 (PD-L1) is an essential inhibitory checkpoint protein, whose aberrant expression plays a key role in cancer immune evasion. 1 Therapeutic blockade of immune checkpoint proteins has reshaped treatment paradigms,2 and thus PD-L1 is commonly tested as a biomarker for immunotherapy eligibility.Recently, deep learning models have been employed to predict protein biomarker expression using haematoxylin and eosin (H&E) images.3 Conventionally, multi-class models are used to assess prediction classes, however, the challenge of correctly predicting new instances increases concomitantly with the number of classes, hindering model performance.4 In this study, we propose a unique pipeline employing two model architectures, Pix2Pix generative adversarial network (GAN)5 and vision transformer (ViT),6 to accurately predict PD-L1 within cytokeratin (CK) positive cell populations.Methods Formalin-fixed paraffin-embedded lung carcinoma tissue sections (4µm) (n=24) were subjected to multiplex immunohistochemical (mIHC) staining (pan-CK and PD-L1), counterstained with DAPI, and imaged (fluorescence) using the Zeiss AxioScan 7 (Zeiss, Germany). The same section then underwent H&E staining, and was imaged (brightfield) using the same scanner. Following StarDist segmentation, 7 mIHC images were registered to their H&E counterparts using nuclear stains. Cell-level ground truth labels for CK and PD-L1 were obtained from their respective mIHC channels and mapped to the H&E images. Whole slide H&E images and CK masks were then divided into tiles (512 x 512-pixels). Balanced training and validation datasets were created (80:20 split) from the tiles of 20 whole slide H&E images. These images were then used to train a Pix2Pix-GAN model5 to differentiate between CK+ and CK- signals. Four cases (92,637 tiles) were set aside as a held-out test dataset. Cells with CK+ ground truth labels were further divided into single cell image patches (112 x 112-pixels) to train a vision transformer model (ViT-B/16)6 for PD-L1 binary classification (figure 1).Results Our Pix2Pix GAN model accurately predicted CK positivity (F1 score: 0.6890; figure 2A-B). Similarly, ViT binary classification demonstrated efficient differentiation between CK+PD-L1- and CK+PD-L1+ cells from within the tumour population (F1 score: 0.6303, AUROC: 0.6561; figure 2C-E).Conclusions Our novel pipeline allows for modularization, facilitating biomarker-specific hyperparameter tuning. This gives rise to increased training flexibility and accurate biomarker prediction. Through efficient deep learning prediction of immunotherapy biomarkers, it may be possible to streamline patient triage within the clinical setting, resulting in reduced resource burden and improved patient waiting time. Future works refining and improving model performances are ongoing.Acknowledgements We thank our institutions for supporting this work.References Lee D, Cho M, Kim E, et al. PD-L1: from cancer immunotherapy to therapeutic implications in multiple disorders. Molecular Therapy. 2024;32(12):4235–55.Cheng W, Kang K, Zhao A, et al. Dual blockade immunotherapy targeting PD-1/PD-L1 and CTLA-4 in lung cancer. J Hematol Oncol. 2024;17(1):54.Niehues JM, Quirke P, West NP, et al. Generalizable biomarker prediction from cancer pathology slides with self-supervised deep learning: a retrospective multi-centric study. Cell Reports Medicine. 2023;4(4):100980.Moral PD, Nowaczyk S, Pashami S. Why is multiclass classification hard? IEEE Access. 2022;10:80448–62.Isola P, Zhu J-Y, Zhou T, et al. Image-to-image translation with conditional adversarial networks. 2017:5967–76 .Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint arXiv:201011929. 2020.Schmidt U, Weigert M, Broaddus C, et al. editors. Cell detection with star-convex polygons. medical image computing and computer assisted intervention – MICCAI 2018; 2018 2018//; Cham: Springer International Publishing.Ethics Approval This study was approved by the Agency for Science, Technology and Research; IRB number 2021-188.Abstract 1089 Figure 1Workflow of dual model framework. A) H&E images are segmented. mIHC channels are registered to the H&E before extracting tiles and cell patches. B) Cytokeratin positive cells are identified by a Pix2Pix GAN model. C) PD-L1 is predicted from CK+ tumour cells using a ViT-B/16 modelAbstract 1089 Figure 2Pix2Pix GAN and ViT binary classifier model performance. A) Confusion matrix for Pix2Pix GAN model (n = 4). B) Representative input and generated images from the Pix2Pix GAN. C) AUROC for ViT-B/16 model (n = 4). D) Confusion matrix for the ViT-B/16 model (n = 4). E) H&E images used within the ViT-B/16 model",
  "authors": [
    {
      "affiliations": [
        "Agency for Science, Technology and Research, Singapore, Singapore"
      ],
      "name": "Rachel EA Fincham"
    },
    {
      "affiliations": [
        "Agency for Science, Technology and Research, Singapore, Singapore"
      ],
      "name": "Craig Joseph"
    },
    {
      "affiliations": [
        "IMCB/A*STAR, Singapore, Singapore"
      ],
      "name": "Felicia Wee"
    },
    {
      "affiliations": [
        "Agency for Science, Technology and Research, Singapore, Singapore"
      ],
      "name": "Ruisi Li"
    },
    {
      "affiliations": [
        "National University of Singapore, Singapore, Singapore"
      ],
      "name": "Jiangfeng Ye"
    },
    {
      "affiliations": [
        "National University of Singapore, Singapore, Singapore"
      ],
      "name": "Lanqi Nie"
    },
    {
      "affiliations": [
        "Agency for Science, Technology and Research, Singapore, Singapore"
      ],
      "name": "Timothy O Wang"
    },
    {
      "affiliations": [
        "Singapore General Hospital, Singapore, Singapore"
      ],
      "name": "Joe Poh Sheng Yeong"
    }
  ],
  "title": "1089 H&E 2.0: deep learning prediction of lung cancer biomarkers pan-cytokeratin and PD-L1 using a dual model framework",
  "uid": "f873b429-5934-5236-8aab-32ce294d12fd"
}