{
  "abstract": "Objective Early detection of large vessel occlusion (LVO) on non-contrast CT (NCCT) could accelerate stroke triage, but NCCT based artificial intelligence (AI) algorithms lack multinational validation and evidence of impact on clinician performance. We aimed to validate a machine learning based LVO detection algorithm across multinational cohorts and evaluate its impact on clinicians’ diagnostic performance.Methods In this retrospective study, an AI algorithm was validated using independent cohorts from Korea (n=723; consecutive; 127 with LVO) and the US (n=240; case-control; 120 with LVO). Standalone performance was assessed using the area under the receiver operating characteristic curve (AUC), sensitivity, and specificity. A multi-reader, multi-case crossover study involving eight physicians evaluated diagnostic performance with and without AI assistance. Clinical utility was quantified using net reclassification improvement (NRI), number needed to screen (NNS), and benefit-to-harm ratio (BHR).Results Standalone AI achieved AUC values of 0.963 (95% CI 0.946 to 0.975) in the Korean cohort and 0.899 (95% CI 0.858 to 0.939) in the US cohort. AI assistance significantly increased pooled AUC values from 0.718 to 0.852, sensitivity from 46.6% to 63.7%, and specificity from 91.9% to 94.9% (all P<0.001). AI gave an NRI of 5.5%, an NNS of 18.2, and a BHR of 2.89. Automation bias analysis showed a reliance level BHR of 9.25.Conclusion This multinational validation confirmed the robustness and generalizability of a machine learning based LVO detection algorithm on NCCT. AI assistance significantly improved clinicians’ sensitivity and diagnostic accuracy, suggesting its potential as a supportive tool in acute stroke triage.",
  "authors": [
    {
      "affiliations": [
        "Department of Radiology, Seoul National University Bundang Hospital, Seoul National University College of Medicine, Seongnam, Korea (the Republic of)"
      ],
      "name": "Leonard Sunwoo"
    },
    {
      "affiliations": [
        "Artificial Intelligence Research Center, JLK, Seoul, Korea (the Republic of)"
      ],
      "name": "Wi-Sun Ryu"
    },
    {
      "affiliations": [
        "Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts, USA"
      ],
      "name": "Karen Buch"
    },
    {
      "affiliations": [
        "Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts, USA"
      ],
      "name": "John Conklin"
    },
    {
      "affiliations": [
        "Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts, USA"
      ],
      "name": "William Mehan"
    },
    {
      "affiliations": [
        "Department of Radiology, Brigham and Women’s Hospital, Boston, Massachusetts, USA"
      ],
      "name": "Matthew Nicholas Desalvo"
    },
    {
      "affiliations": [
        "Department of Radiology, Brigham and Women’s Hospital, Boston, Massachusetts, USA"
      ],
      "name": "Erik Loken"
    },
    {
      "affiliations": [
        "Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts, USA"
      ],
      "name": "Sandra Rincon"
    },
    {
      "affiliations": [
        "Department of Radiology, Brigham and Women’s Hospital, Boston, Massachusetts, USA"
      ],
      "name": "Tatiana Rocha"
    },
    {
      "affiliations": [
        "Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts, USA"
      ],
      "name": "Marc Succi"
    },
    {
      "affiliations": [
        "Artificial Intelligence Research Center, JLK, Seoul, Korea (the Republic of)"
      ],
      "name": "Myungjae Lee"
    },
    {
      "affiliations": [
        "Artificial Intelligence Research Center, JLK, Seoul, Korea (the Republic of)"
      ],
      "name": "Dongmin Kim"
    },
    {
      "affiliations": [
        "Department of Neurology and Cerebrovascular Center, Seoul National University Bundang Hospital, Seoul National University College of Medicine, Seongnam, Korea (the Republic of)"
      ],
      "name": "Nak-Hoon Kim"
    },
    {
      "affiliations": [
        "Department of Neurology, Korea University Guro Hospital, Seoul, Korea (the Republic of)"
      ],
      "name": "Jung-Hoon Han"
    },
    {
      "affiliations": [
        "Department of Neurology, Korea University Guro Hospital, Seoul, Korea (the Republic of)"
      ],
      "name": "Kyubong Lee"
    },
    {
      "affiliations": [
        "Department of Radiology, Seoul National University Bundang Hospital, Seoul National University College of Medicine, Seongnam, Korea (the Republic of)"
      ],
      "name": "Younbeom Jeong"
    },
    {
      "affiliations": [
        "Department of Neurology, Korea University Guro Hospital, Seoul, Korea (the Republic of)"
      ],
      "name": "Hyesu Yun"
    },
    {
      "affiliations": [
        "Department of Radiology, Seoul National University Bundang Hospital, Seoul National University College of Medicine, Seongnam, Korea (the Republic of)"
      ],
      "name": "Gwanhu Kim"
    },
    {
      "affiliations": [
        "Department of Neurology and Cerebrovascular Center, Seoul National University Bundang Hospital, Seoul National University College of Medicine, Seongnam, Korea (the Republic of)"
      ],
      "name": "Byeongcheon Lee"
    },
    {
      "affiliations": [
        "Department of Neurology and Cerebrovascular Center, Seoul National University Bundang Hospital, Seoul National University College of Medicine, Seongnam, Korea (the Republic of)"
      ],
      "name": "Yoona Ko"
    },
    {
      "affiliations": [
        "Department of Radiology, Seoul National University Bundang Hospital, Seoul National University College of Medicine, Seongnam, Korea (the Republic of)"
      ],
      "name": "Cheolho Kwon"
    },
    {
      "affiliations": [
        "Department of Radiology, Seoul National University Hospital, Seoul National University College of Medicine, Seoul, Korea (the Republic of)"
      ],
      "name": "Roh-Eul Yoo"
    },
    {
      "affiliations": [
        "Department of Neurology, Northwestern University Feinberg School of Medicine, Chicago, Illinois, USA"
      ],
      "name": "Philip Gorelick"
    },
    {
      "affiliations": [
        "Department of Neurology and Cerebrovascular Center, Seoul National University Bundang Hospital, Seoul National University College of Medicine, Seongnam, Korea (the Republic of)"
      ],
      "name": "Hee-Joon Bae"
    },
    {
      "affiliations": [
        "Department of Neurology, Korea University Guro Hospital, Seoul, Korea (the Republic of)"
      ],
      "name": "Chi Kyung Kim"
    },
    {
      "affiliations": [
        "Department of Neurology and Cerebrovascular Center, Seoul National University Bundang Hospital, Seoul National University College of Medicine, Seongnam, Korea (the Republic of)"
      ],
      "name": "Beom Joon Kim"
    }
  ],
  "title": "AI assisted detection of large vessel occlusion on non-contrast CT: multinational validation and reader study",
  "uid": "fed0e402-403a-51e6-8907-9e205d75e46d"
}