{
  "abstract": "Background Discharge summaries are a vital piece of clinical documentation, however, they occupy a significant portion of the junior clinicians workload. The efficacy of large language models (LLMs) in discharge summary preparation using real clinical documentation remains novel. Integrating LLMs into discharge summary production workflows in Neurology may help reduce clinical administrative burdons for junior clinicians.Methods Our study aimed to test the efficacy of two, open-sourced LLMs to generate DC summaries which were scored using a validated DC summary scoring metric. Retrospectively the clinician-generated discharge summaries were also scored.Results The models performed nearly identically, with the llama3:instruct model having a mean score of 19.1/31 (SD: 2.42) compared to 19.2/31 (SD: 3.48) when produced by llama3:70b. When compared to the scores for the human written discharge summaries (mean score: 24.2/31, SD: 3.32), the LLMs both underperformed (P=0.0241). Key areas of underperformance were mainly attributed to limitations in LLM context-window length.Conclusion Although the total scores for each LLM were below that of expected discharge summary quality, even with a rudimentary and basic prompt, both models were able to deliver a reasonable result that, with minor edits and inclusions, would make a sufficient discharge summary. Mixed solutions and workflows which combine the summarisation and information synthesis capabilities of LLMs with clinican oversight for fields which require precision (e.g. medication lists) may reduce the amount of time junior clinicans spend generating dischare summaries. Prompts may also be adapated to different specialties (such as Neurology), to ensure integration of relevent clinical data.",
  "authors": [
    {
      "affiliations": [
        "Adelaide Medical School, University of Adelaide, SA, Australia"
      ],
      "name": "Lewis Hains"
    },
    {
      "affiliations": [
        "Adelaide Medical School, University of Adelaide, SA, Australia"
      ],
      "name": "Oliver Kleinig"
    },
    {
      "affiliations": [
        "Adelaide Medical School, University of Adelaide, SA, Australia"
      ],
      "name": "Ashwin Murugappa"
    },
    {
      "affiliations": [
        "Adelaide Medical School, University of Adelaide, SA, Australia",
        "Lyell McEwin Hospital, Elizabeth Vale, SA, Australia"
      ],
      "name": "Samuel Gluck"
    },
    {
      "affiliations": [
        "Lyell McEwin Hospital, Elizabeth Vale, SA, Australia"
      ],
      "name": "Jarrod Marks"
    },
    {
      "affiliations": [
        "Adelaide Medical School, University of Adelaide, SA, Australia",
        "Lyell McEwin Hospital, Elizabeth Vale, SA, Australia"
      ],
      "name": "Toby Gilbert"
    },
    {
      "affiliations": [
        "Adelaide Medical School, University of Adelaide, SA, Australia",
        "Harvard University, Boston, Massachusetts, USA",
        "Massachusetts General Hospital, Boston, Massachusetts, USA"
      ],
      "name": "Stephen Bacchi"
    }
  ],
  "title": "3593 Large language models aiding efficient discharge summary preparation",
  "uid": "e46a9e0a-4414-59eb-bb2b-85a9ed850b53"
}