Show download pdf controls
  • Reliability assessment

    All gap estimates are assessed for reliability against 10 criteria. The reliability rating provides a transparent assessment of our gap estimates, drawing on the International Monetary Fund (IMF) and our expertise. We summarise this in a rating assessment for each gap estimate.

    In assessing reliability, we provide the expert panel with our initial views and working material. The expert panel assesses each submission and provides feedback for improvements. Once all feedback is addressed, the panel endorses a final rating.

    Reliability criteria

    The ten reliability criteria are considered of equal importance. We sort them into three groups by evaluation stage, as follows:

    • Evaluation of the estimation framework    
      • capture the appropriate tax base
      • cover all potential taxpayers
      • account for all potential forms of non-compliance
      • no overlap within or between any components of the framework.
       
    • Evaluation of the methodology    
      • evaluate the approach used against the assessment criteria for that methodology
      • ensure the most appropriate method is used and results are validated against supporting information
      • sensitivity to the underlying model, assumption and structure
      • assessment of assumptions, judgment or expertise.
       
    • Evaluation of the internal process and delivery    
      • evaluate the quality of the management process
      • the analysis provides insights into the drivers of a gap estimate.
       

    Reliability ratings

    For each estimate, each reliability criterion is scored from 0 ('poor or missing') to 30 ('excellent'). The sum of these scores determines the reliability rating.

    The total reliability score ranges from 0 to 30 and is placed into one of five categories, as follows:

    • Very low (a score of 0 to 10) – the results are preliminary or interim in nature, often being a pilot estimate in its first years of production. The estimate may have a number of issues that compromise its reliability. It may have a large or unknown margin of error. The estimate is not confirmed by other independent analysis. The results provide very little information and could be misleading.
    • Low (a score of 11 to 15) – a large number of factors are not considered in the estimate. The estimate has a material margin of error. The estimate may be partially confirmed by other analyses. It should not be used to provide insight into population compliance. It may, however, provide direction for further research and analysis. Improvements, when made, may significantly alter the gap estimate.
    • Medium (a score of 16 to 20) – a number of factors are not considered which, if addressed, may change gap estimates to a limited or immaterial degree. The estimate has an acceptable margin of error. The estimate is derived from an appropriate calculation methodology. It is materially confirmed by other analyses, such as risk models and intelligence scans. With caution and contextualisation it can provide insight into population compliance.
    • High (a score of 21 to 25) – a small number of factors are not considered which, if addressed, may change the gap estimate to a very limited or immaterial degree. The estimate has a low margin of error. The estimate is derived from a highly appropriate calculation methodology. It is materially confirmed by other analyses such as risk models and intelligence scans. With caution and contextualisation, it can provide insight into population compliance.
    • Very high (a score of 26 to 30) – all factors are considered. The estimate has a very low margin of error. The estimate is derived from the most appropriate calculation methodology. It is confirmed by other analyses, such as risk models and intelligence scans. It can provide highly detailed insights into the levels of compliance across the population.

    Return to:

      Last modified: 19 Oct 2020QC 53161