{
  "abstract": "Objectives To develop and validate a tool for standardised quality assessment of data-driven algorithms in healthcare, focusing on the underlying data pipeline.Methods Data Assessment Tool for Algorithm Critical Appraisal and Robust Evidence (DATA-CARE) was iteratively developed from the established Quality In Prognosis Studies framework, selected after reviewing 10 existing quality assessment tools for observational and artificial intelligence studies. DATA-CARE evaluates five quality domains of the data pipeline: study population, data, algorithm, outcome and report transparency. Each domain comprises three to five quality criteria. With a total score of 75 points, study quality is categorised as low (<45), moderate (45–59) or high (≥60). DATA-CARE was validated during a systematic review on data-driven algorithms using continuous physiological monitoring data within the paediatric intensive care unit. Two independent reviewers performed quality assessment using DATA-CARE of included studies. Tool validation was evaluated using inter-rater agreement and intraclass correlation coefficient (ICC).Results DATA-CARE demonstrated robust inter-rater agreement (93.5%) with ICC 0.98 (95% CI 0.96 to 0.99). Of 3858 screened studies, 31 were reviewed in the use case, describing diverse algorithms. Studies were predominantly low (32.3%) to moderate (41.9%) and sporadically (25.8%) high quality.Discussion Predominance of low-to-moderate quality studies reveals critical barriers to clinical implementation of data-driven algorithms, including low quality data capture and processing, lacking validation strategies and non-transparent reporting of findings.Conclusions DATA-CARE allows standardised and reliable critical appraisal for a wide variety of algorithms, addressing current gaps in standardised and reproducible algorithm development.",
  "authors": [
    {
      "affiliations": [
        "Department of Neonatal and Pediatric Intensive Care, Division of Pediatric Intensive Care, Erasmus MC, Rotterdam, The Netherlands"
      ],
      "name": "Eris van Twist"
    },
    {
      "affiliations": [
        "Department of Neonatal and Pediatric Intensive Care, Division of Pediatric Intensive Care, Erasmus MC, Rotterdam, The Netherlands"
      ],
      "name": "Brian van Winden"
    },
    {
      "affiliations": [
        "Department of Neonatal and Pediatric Intensive Care, Division of Pediatric Intensive Care, Erasmus MC, Rotterdam, The Netherlands"
      ],
      "name": "Rogier de Jonge"
    },
    {
      "affiliations": [
        "Department of Neonatal and Pediatric Intensive Care, Division of Pediatric Intensive Care, Erasmus MC, Rotterdam, The Netherlands"
      ],
      "name": "H Rob Taal"
    },
    {
      "affiliations": [
        "Department of Neonatal and Pediatric Intensive Care, Division of Pediatric Intensive Care, Erasmus MC, Rotterdam, The Netherlands"
      ],
      "name": "Matthijs de Hoog"
    },
    {
      "affiliations": [
        "Department of Biomechanical Engineering, Faculty of Mechanical Engineering, Delft University of Technology, Delft, The Netherlands"
      ],
      "name": "Alfred Schouten"
    },
    {
      "affiliations": [
        "Pattern Recognition Laboratory, Faculty of Electrical Engineering, Mathematics and Computer Science, Delft University of Technology, Delft, The Netherlands"
      ],
      "name": "David Tax"
    },
    {
      "affiliations": [
        "Department of Neonatal and Pediatric Intensive Care, Division of Pediatric Intensive Care, Erasmus MC, Rotterdam, The Netherlands"
      ],
      "name": "Jan Willem Kuiper"
    }
  ],
  "title": "Data pipeline quality: development and validation of a quality assessment tool for data-driven algorithms and artificial intelligence in healthcare",
  "uid": "1583ed78-7da5-5b46-b8eb-3ad79f8dc00c"
}