{
  "abstract": "Introduction The use of artificial intelligence (AI) in healthcare is growing, yet its role in paediatric pharmacy remains uncertain. 1 2 Clinical decision-making in paediatrics often relies on expert interpretation and adaptation, given the lack of robust evidence or licensed medicines in many areas. The Neonatal and Paediatric Pharmacy Group (NPPG) have a message board where members can share information and seek peer input, including advice on complex clinical queries. This study aimed to assess the accuracy of four widely available AI models - ChatGPT-4, Claude, Gemini 2.0 Flash, and Perplexity Deep Research - in responding to these real-world, complex paediatric pharmacy questions.Method Twenty anonymised clinical queries were selected from the NPPG message board. These were chosen for their complexity and likelihood of requiring input beyond standard reference sources. Responses were obtained from the four AI models using a standardised prompt and were also independently answered by senior paediatric pharmacists. A consensus gold-standard answer for each question was created based on expert responses and best available evidence.Each AI response was assessed using a scoring framework across four domains: accuracy, relevance, comprehensiveness, and practicality. Scores ranged from 0 (no mark) to 3 (high) per domain, with a maximum possible score of 12 per response. Six paediatric pharmacists independently scored the AI responses, with any discrepancies resolved through discussion.Results The four AI models demonstrated varied performance across the twenty clinical scenarios. Four answers were thought to have the potential to cause patient harm (across all models except Claude). ChatGPT-4 consistently achieved the highest scores, particularly for intermediate and complex questions, with structured answers and supporting references.Perplexity performed well on simpler questions and where answers could be readily found online, but was less reliable on more complex or paediatric-specific content. Claude and Gemini frequently often returned generic responses lacking sufficient clinical detail.Across all models, common limitations included outdated or missing references, insufficient consideration of paediatric-specific issues, and a lack of practical detail. While AI tools sometimes aligned with the expert consensus, discrepancies often arose in queries involving off-label use, drug manipulation, or paediatric dosing adjustments.On straightforward questions, such as administration via feeding tubes or dosing for common conditions, all models provided broadly acceptable answers. However, only Perplexity demonstrated a consistent ability to reference current guidance and provide structured responses for more complex topics.Conclusion AI models show promise in supporting paediatric pharmacy decision-making, particularly for straightforward queries. However, their ability to address more complex or nuanced clinical questions remains limited. ChatGPT-4 was the most consistent performer overall, but all models demonstrated important gaps in accuracy, paediatric applicability, and referencing. Human expertise remains essential, particularly when dealing with high-risk decisions or areas where guidance is lacking. Further work is needed to improve the clinical reliability and paediatric relevance of AI tools before they can be safely embedded into pharmacy practice.References Nagy M, Sisk B, Lai A, Kodish E. Will artificial intelligence widen the therapeutic gap between children and adults? Pediatr Investig 2023;8:1–6.Wilhelm C, Steckelberg A, Rebitschek FG. Benefits and harms associated with the use of AI-related algorithmic decision-making systems by healthcare professionals: a systematic review. Lancet Reg Health Eur 2025;48:101145.",
  "authors": [
    {
      "affiliations": [
        "Evelina London Children’s Hospital, Guy’s and St Thomas’ NHS Foundation Trust, UK"
      ],
      "name": "Ashifa Trivedi"
    },
    {
      "affiliations": [
        "Barts Health NHS Trust, UK"
      ],
      "name": "Chloe Benn"
    },
    {
      "affiliations": [
        "Evelina London Children’s Hospital, Guy’s and St Thomas’ NHS Foundation Trust, UK"
      ],
      "name": "Patrick To"
    },
    {
      "affiliations": [
        "Great Ormond Street Hospital for Children NHS Foundation Trust, UK"
      ],
      "name": "Sadhna Sharma"
    },
    {
      "affiliations": [
        "Manchester University NHS Foundation Trust, UK"
      ],
      "name": "Chris Paget"
    },
    {
      "affiliations": [
        "Barts Health NHS Trust, UK"
      ],
      "name": "Mohammed Abou Daya"
    }
  ],
  "title": "SP2 How good is AI? Comparison of models for paediatric pharmacy clinical questions",
  "uid": "4c1504a8-dff0-5cff-bf8e-c341236f06fd"
}