{
  "abstract": "Aims To investigate rule-based and deep learning (DL)-based methods for the automatically generating natural language diagnostic reports for macular diseases.Methods This diagnostic study collected the ophthalmic images of 2261 eyes from 1303 patients. Colour fundus photographs and optical coherence tomography images were obtained. Eyes without retinal diseases as well as eyes diagnosed with four macular diseases were included. For each eye, a diagnostic report was written with a format consisting of lesion descriptions, diagnoses and recommendations. Subsequently, a rule-based natural language processing (NLP) and a DL-based NLP system were developed to automatically generate a diagnostic report. To assess the effectiveness of these models, two junior ophthalmologists wrote diagnostic reports for the collected images independently. A questionnaire was designed and judged by two retina specialists to grade each report’s readability, correctness of diagnosis, lesion description and recommendations.Results The rule-based NLP reports achieved higher grades over junior ophthalmologists in correctness of diagnosis (9.13±1.52 vs 9.03±1.42 points) and recommendations (8.55±2.74 vs 8.50±2.53 points). Furthermore, the DL-based NLP reports got slightly lower grades to those of junior ophthalmologists in lesion description (8.82±1.84 vs 9.12±1.20 points, p<0.05), correctness of diagnosis (8.72±2.36 vs 9.08±1.55 points, p<0.05) and recommendations (8.81±2.52 vs 9.15±1.65 points, p<0.05). For readability, the DL-based reports performed better than junior ophthalmologists, with scores of 9.98±0.17 vs 9.94±0.25 points (p=0.094).Conclusions The multimodal AI system, coupled with the NLP algorithm, has demonstrated competence in generating reports for four macular diseases compared with junior ophthalmologists.",
  "authors": [
    {
      "affiliations": [
        "Department of Ophthalmology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences, Peking Union Medical College, Beijing, China",
        "Beijing Key Laboratory of Fundus Diseases Intelligent Diagnosis & Drug/Device Development and Translation, Beijing, China",
        "Key Laboratory of Ocular Fundus Diseases, Chinese Academy of Medical Sciences, Beijing, China"
      ],
      "name": "Xufeng Zhao"
    },
    {
      "affiliations": [
        "Department of Ophthalmology, Dalian No.3 People’s Hospital, Dalian, Liaoning, China"
      ],
      "name": "Chunshi Li"
    },
    {
      "affiliations": [
        "Department of Ophthalmology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences, Peking Union Medical College, Beijing, China",
        "Beijing Key Laboratory of Fundus Diseases Intelligent Diagnosis & Drug/Device Development and Translation, Beijing, China",
        "Key Laboratory of Ocular Fundus Diseases, Chinese Academy of Medical Sciences, Beijing, China"
      ],
      "name": "Jingyuan Yang"
    },
    {
      "affiliations": [
        "Department of Ophthalmology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences, Peking Union Medical College, Beijing, China",
        "Beijing Key Laboratory of Fundus Diseases Intelligent Diagnosis & Drug/Device Development and Translation, Beijing, China",
        "Key Laboratory of Ocular Fundus Diseases, Chinese Academy of Medical Sciences, Beijing, China"
      ],
      "name": "Xingwang Gu"
    },
    {
      "affiliations": [
        "Department of Ophthalmology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences, Peking Union Medical College, Beijing, China",
        "Beijing Key Laboratory of Fundus Diseases Intelligent Diagnosis & Drug/Device Development and Translation, Beijing, China",
        "Key Laboratory of Ocular Fundus Diseases, Chinese Academy of Medical Sciences, Beijing, China"
      ],
      "name": "Bing Li"
    },
    {
      "affiliations": [
        "Department of Ophthalmology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences, Peking Union Medical College, Beijing, China",
        "Beijing Key Laboratory of Fundus Diseases Intelligent Diagnosis & Drug/Device Development and Translation, Beijing, China",
        "Key Laboratory of Ocular Fundus Diseases, Chinese Academy of Medical Sciences, Beijing, China"
      ],
      "name": "Yuelin Wang"
    },
    {
      "affiliations": [
        "Department of Ophthalmology, Hunan Provincial People’s Hospital, Changsha, Hunan, China"
      ],
      "name": "Bi-lei Zhang"
    },
    {
      "affiliations": [
        "Key Lab of DEKE, Renmin University of China, Beijing, China"
      ],
      "name": "Xirong Li"
    },
    {
      "affiliations": [
        "Vistel AI Lab, Visionary Intelligence Ltd, Beijing, China"
      ],
      "name": "Jianchun Zhao"
    },
    {
      "affiliations": [
        "Vistel AI Lab, Visionary Intelligence Ltd, Beijing, China"
      ],
      "name": "Jie Wang"
    },
    {
      "affiliations": [
        "Department of Ophthalmology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences, Peking Union Medical College, Beijing, China",
        "Beijing Key Laboratory of Fundus Diseases Intelligent Diagnosis & Drug/Device Development and Translation, Beijing, China",
        "Key Laboratory of Ocular Fundus Diseases, Chinese Academy of Medical Sciences, Beijing, China"
      ],
      "name": "Weihong Yu"
    }
  ],
  "title": "Diagnostic report generation for macular diseases by natural language processing algorithms",
  "uid": "d170c522-4856-5cd4-b5a0-faf9fcd516e7"
}