{
  "abstract": "Inconsistency is a key domain that determines the certainty of evidence. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach specifically defines inconsistency as the variability in results across studies, and not variability in study characteristics, eligibility criteria or design. 1 Statistical measures of heterogeneity are often used to assess inconsistency, however, major limitations of such measures have been described. For example, Cochran’s Q test for homogeneity is usually underpowered to detect heterogeneity. The I2 index which is the most commonly used measure, underestimate true statistical heterogeneity when there are fewer than 10 studies in a meta-analysis, which is a common scenario, and is correlated with the sample size of the included studies.2 The I2 index is also often misunderstood as an indicator of the spread of the effect size. Borenstein demonstrates how a meta-analysis with I2 index of 25% can have more spread of the effect size than a meta-analysis with I2 index of 75%.3 Therefore, GRADE guidance on inconsistency recommended less reliance on statistical measures and instead, instructed to make judgements about whether studies in a meta-analysis provide estimates that are clinically importantly different from each other.1 However, there are no existing tools to facilitate this process making it highly subjective. Users are instructed to look at a forest plot and evaluate the similarity of point estimates of the included studies and the overlap of their CIs, and make a judgement based on values that they consider clinically important. Merely counting studies does not work because some studies can be outliers but may have a very small weight within the pooled effect estimate. Having multiple thresholds makes this task even more difficult. Furthermore, in the case of binary outcomes, decision thresholds are based on absolute treatment effects4 5 whereas most meta-analyses and their associated forest plot are performed on relative effect scales.",
  "authors": [
    {
      "affiliations": [
        "Evidence-based Practice Center, Mayo Clinic, Rochester, MN, USA"
      ],
      "name": "Mohammad Hassan Murad"
    },
    {
      "affiliations": [
        "Evidence-based Practice Center, Mayo Clinic, Rochester, MN, USA",
        "Health Care Policy and Research, Mayo Clinic Minnesota, Rochester, Minnesota, USA"
      ],
      "name": "Zhen Wang"
    },
    {
      "affiliations": [
        "Case Western Reserve University, Cleveland, Ohio, USA"
      ],
      "name": "Yngve Falck-Ytter"
    }
  ],
  "title": "Facilitating GRADE judgements about the inconsistency of effects using a novel visualisation approach",
  "uid": "cebd9a4a-af1d-5db3-9ed5-0f117bd6b6aa"
}