Approaches to estimate tax gaps
We typically use 2 broad approaches to estimate tax gaps – the top-down or bottom-up approaches (shown in Figure 4).
- Top-down approaches use externally provided aggregated data sources to estimate the size of the tax base, from which we estimate the theoretical tax liability. The difference between the theoretical tax liability and the amount we receive is the estimated tax gap. A top-down approach is typically used for indirect taxes.
- Bottom-up approaches involve a detailed examination of data sources, such as tax returns, audit results (including random enquiry programs), risk registers or third-party data-matching information. We use this information to determine the extent of non-compliance across the whole population, from which we estimate the tax gap. A bottom-up approach is typically used for direct taxes. There are 3 types as shown in Figure 5 and described further below.
Figure 3: Our 2 approaches to estimate tax gaps
Figure 4: Our tax gaps by approach
Choosing the right method
We choose the method that provides the most reliable estimate for each gap we measure. In doing this, we carefully consider the characteristics of each gap, including:
- the design of the tax or program
- the characteristics of the population
- availability and quality of data.
Assessing these factors helps us decide which method is the most appropriate to use. For example, to use a top-down method we generally require external data. If we don't have a reliable external data source available, we know we'll need to use a bottom-up method to generate a reliable result.
We assess our methodologies for reliability, and where possible test them against alternatives to ensure that we are using the most appropriate method. We consult with our engagement, advice and assurance on the options available to us. We also look to other jurisdictions to see which methodologies they use for similar gaps.
We continually work to update and improve our gap estimates. Part of this involves assessing the method used to ensure it's still the most appropriate option. This means we can remain confident that our gap estimates are reliable and credible.
Gap approaches in detail
This section provides a more detailed explanation of the methods we use in top-down and bottom-up approaches to measure tax gap estimates.
Top-down methods
To implement a top-down approach, the tax system is first conceptualised as part of the economy, and a method is developed to break the system down to understand each of its constituent parts and how these work individually. Top-down methods use external information about how economic activity and the tax system interact.
This method doesn't always provide information on what drives the tax gap but rather tells us that a gap exists. An example of this is the goods and services tax (GST) gap, which uses information collected through the Australian National Accounts data set. This data is collated by the Australian Bureau of Statistics (ABS) and covers detailed estimates of different forms of goods and services that are produced in Australia. We can use that data to estimate the GST that should be collected on each different category of goods and services and aggregate that to total theoretical estimate and compare it to the GST that is actually collected.
Bottom-up methods
We use 3 broad types of method to implement bottom-up approaches:
Random enquiry programs
A random enquiry program (REP) is a process for selecting taxpayers, tax returns, or transactions for evaluation. As the name suggests, the taxpayer, tax return, or transaction is randomly selected. This ensures that each unit in the population of interest has the same likelihood of being chosen.
This is unlike operational audit selection processes, which focus on taxpayers, tax returns, or transactions considered to present a risk to tax or compliance outcomes. Operational audit data is biased towards higher likelihood, higher consequence segment of taxpayers, returns or transactions. These means the results from operational audits are typically not representative of compliance risks amongst those that were not selected for audit.
In contrast, random selection avoids any systematic selection of segments of the population, returns or transactions. Randomisation is designed to provide an unbiased sample that is representative of overall outcomes in the system.
Statistical-based approaches
Statistical-based approaches use a set of mathematical models to estimate an outcome where it would be impractical to obtain a data set that covers 100% of the population being estimated or a representative random sample. This typically means drawing on operational audit data and correcting for selection bias.
The suitability of any statistical method depends on the characteristics of the population being measured. As methods, data and insights evolve, we continue to review and refine our approaches to ensure they remain appropriate and produce reliable estimates.
We currently use 2 types of statistical-based approaches within the tax gap program to estimate the various tax gaps:
Regression analysis
Regression analysis is a standard statistical technique for estimating the relationships between one variable and a series of other variables. The regression can be used to identify the probability or the magnitude of the tax gap using all available taxpayer records and compliance results.
To produce reliable and credible results when using regression analysis, corrections need to be made for selection bias. This bias exists because taxpayers we undertake our compliance activities on are higher risk taxpayers. If we don't adjust for this bias, our estimates are likely to be overstated and conclusions misleading. We adjust for selection bias using either:
- propensity score matching
- Heckman’s correction.
The benefit of regression analysis is that it is useful in identifying characteristics that help predict whether a taxpayer is non-compliant, as well as characteristics that help predict the degree of non-compliance. Based on these characteristics, or drivers, the size of the tax gap can be estimated for the taxpayers that are modelled to be non-compliant.
Extreme value theory
Extreme value theory (EVT) is a statistical approach that can be used when data contains a relatively small number of very large observations. For example, a small proportion of cases may account for a significant share of the total compliance revenue.
This type of distribution is commonly seen in finance and physical sciences. We also see similar patterns in tax return amendment data including adjustments arising from taxpayer revisions and compliance activities.
EVT focuses on the relationship between the largest observations in the dataset. These relationships are used to estimate the likely contribution of extreme values across the broader population and inform the final tax gap estimate.
A challenge with EVT is that the presence (or absence) of a small number of very large results can significantly affect estimates year to year. As it is difficult to infer drivers and characteristics from these to the remaining population, such shifts may not represent changes in broader tax compliance.
Model-based approaches
Where random enquiry is not suitable, and available data does not match the assumptions required for a statistical approach, we use model-based approaches.
Model-based approaches identify the key themes, factors or channels that contribute to the gap, which are then used to inform the final estimate. Like all our estimates, they draw on all available data including expert judgment, management information and system data to inform the final estimate.
These approaches can also be individually referred to as:
- micro-analytical simulation
- illustrative
- channel analysis.
The aspect they have in common is a disaggregation, the analysis of known information, then an aggregation to a final estimate.
Model-based approaches can be particularly sensitive to assumptions.