Show download pdf controls
  • Total revenue effects measure – impacts of our engagement activities

    Total revenue effects is one of the indicators we use to monitor and understand the impacts of our activities on the performance and operation of the tax and superannuation systems.

    Total revenue effects is a measure of the impact that ATO activities have on improving taxpayer compliance. It is a combination of both:

    Audit yield

    Audit yield is a historical measure of the collection of specifically identified liabilities raised from our audit and enforcement activities. These liabilities are directly connected to the adjustment we make, and payment can occur after we conduct the audit. It also includes interest and penalties.

    Wider revenue effects

    Wider revenue effects are an estimate of the additional revenue received from clients we influence. They typically represent improved voluntary compliance. When measuring wider revenue effects, we ensure there is a clear connection between our activity and the change in taxpayer behaviour. This connection, and any assumptions that underpin it, must be reasonable and defensible.

    Both audit yield and wider revenue effects are reported on a cash basis rather than accrual basis. This applies to a given ATO activity when the additional revenue is paid by the taxpayer.

    Find out about:

    See also:

    Results

    Our current methods are designed to estimate the revenue effects of improvements to compliance historically, or in following our engagement with clients and their agents (including corrective and preventative work).

    We estimated our 2018–19 total revenue effects to be $15.3 billion.

    Table 1: Total revenue effects, 2016–17 to 2018–19 ($ billion)

    Annual report

    Audit yield
    ($b)

    Wider revenue effects
    ($b)

    Total revenue effects
    ($b)

    2016–17

    10.2

    4.8

    15.0

    2017–18

    11.8

    4.2

    16.0

    2018–19

    10.5

    4.8

    15.3

    Note: Amounts are rounded to the nearest $ million.

    Contributors to total revenue effects

    We focus on improving activity statement lodgment across all client groups.

    We work with individuals and their tax agents to reduce over-claiming of work-related expenses. This includes changing taxpayer behaviour through:

    • real-time nudges through myTax
    • using our data and analytics to stop incorrect claims
    • the emerging benefits of working with agents to address issues that span their client base.

    We also engage with businesses of all sizes to influence correct reporting of income and claiming of deductions including entering into advance pricing arrangements.

    See also:

    Principles for measuring total revenue effects

    We apply four principles for selecting, developing and implementing methods for measuring total revenue effects. These principles represent the foundational propositions that underpin robust and evidence-based measurement.

    Any measurement method should ensure that revenue effects are:

    Attributable

    A revenue effect is considered to be attributable to an action we take when:

    • a case for a causal connection between the ATO action and the revenue effect can be reasonably argued and supported to a level that would withstand independent scrutiny
    • the degree to which the revenue effect can be attributed to the ATO action is established.

    Integrated

    Each method should be developed with regard to other methods to avoid double counting while striving for integrated and comprehensive coverage.

    Defensible

    A method is considered to be defensible if:

    Pragmatic

    A pragmatic approach to performance measurement balances competing demands and the relative value of performance informationExternal Link (page 30). Factors we consider when deciding on measurement methods include:

    • timeliness
    • comprehensiveness
    • cost-effectiveness
    • demonstrating causality and isolating the effect
    • precision and robustness.

    A best-fit approach to measuring revenue effects effectively balances competing factors. Where more definitive methods (like the audit yield approach) are impractical, a statistical method could be suitable, or there could be no cost-effective options at all. It is not necessary for all effects to be definitively measured. However, wherever statistical methods are used, they need to be appropriate, capably implemented, and result in defensible estimates.

    These principles are applicable to measurement methods implemented by business areas across the ATO. They apply regardless of the activity type, characteristics of the tax base or taxpayer population.

    Figure 1: Program logic concept

    Flow chart showing the program logic concept looking at the what the intent is to influence taxpayer behaviour with the relevant ATO activity and what measures of success there is that is immediate and in the medium term along with the long term outcomes.

    Methodologies

    We review methods for estimating wider revenue effects and update them each year with the latest information. We have implemented new and improved techniques, which broaden the coverage of our preventative and corrective taxpayer engagements while maintaining robust and defensible estimates.

    Estimating total revenue effects

    We use two approaches to estimate total revenue effects:

    Primary sources of information

    This is derived from the audit approach to measurement. We calculate the impact the ATO has had on the revenue from each taxpayer by examining the evidence directly. This approach is typically used for audit yield and to estimate the revenue effects for small groups of taxpayers with high incomes and few comparable peers. We often use this method when engaging with large organisations; for instance, we examine an entity's affairs relating to a part of the taxation law for a specified relevant period or periods.

    Statistical estimates

    These estimates are generally based on our analysis of operational data. Methods used are defensible in line with OECD recommendations. These include the use of randomised control trials and other statistical techniques to estimate the impact of our client engagements. Statistical approaches are typically used to estimate the revenue effects for larger groups of taxpayers with lower incomes and many comparable peers.

    Estimating wider revenue effects

    We use the following common statistical methods for estimating wider revenue effects:

    Randomised control trial method

    A randomised control trial (RCT) is one of the simplest and most useful statistical methods. To evaluate the influence of preventative measures on compliance behaviour, we monitor taxpayer behaviour and compare it to clients randomly selected from a quarantined group (known as a control population). These types of preventative measures can include SMS campaigns, pre-emptive communication and early intervention communication.

    For example, to evaluate the influence of an ATO campaign on a specific behaviour, first we would randomly select a group of taxpayers from a defined population of interest (treatment group) and then isolate them from all other ATO actions. We then compare their behaviour to the control population. The effect of the campaign would be determined by comparing the behaviour of the selected group of taxpayers to that of the control population.

    Figure 2 provides an illustration of a sample population and group allocations.

    Figure 2: Sample randomised control trial concept for individual tax return (ITR) late lodgments

    Diagram shows how the overall individual tax return late lodgment population is split into two groups being an intervention and a control group and how then the outcomes from both groups are measured and compared.

    Difference-in-difference method

    The simplest type of difference-in-difference (DiD) method involves comparing the outcomes for two groups (the treated and the control group) for two time periods. The treated group is exposed to an intervention in the second period but not the first. The control group is not exposed to the intervention in either period. The treatment effect is then calculated by taking the difference in the average outcomes of the treated and the control groups over the period (refer to Figure 3).

    Figure 3: Measurement of treatment effects using the difference-in-difference method and parallel trend assumption

    Line graph depicting how the treatment effect is calculated by taking the difference in the average outcomes of the treated and the controls groups over the period.

    Assumptions

    Results are based on the following assumptions:

    • primary assumption of ‘parallel trends’ – the treated and control groups will move with the same trends in the absence of intervention
    • secondary assumption – that most differences in the characteristics between treated and control are time-invariant.

    When these assumptions are met this method will remove most sources of selection bias.

    When identifying a control group, there are likely to be a number of time-varying characteristics amongst the companies assigned to either of the groups. This leads to a high likelihood of the data failing the parallel trends assumption (refer to Figure 4).

    Figure 4: Lack of parallel trends in actual data

    Line graph comparing the treated group vs the untreated group based on a time variant and how the “parallel trends” assumption fails.

    One technique to satisfy the parallel trends assumption is to combine the characteristics of individual untreated cases on a weighted average basis. This is done in such a way that their average outcomes mimic that of the treated cases in the pre-intervention period (refer to Figure 5). This becomes the control.

    Figure 5: After the construction of the control from weighted 'near neighbours'

    Line graph illustrating the technique used to satisfy the parallel trends assumption by combining the characteristics of individual untreated cases on a weighted average basis. This is done in such a way that their average outcomes mimic that of the treated cases in the pre-intervention period.

    In the post-treatment period, the same weights are applied to the actual tax outcomes of the companies in the control group. This helps construct a counterfactual position against which the outcome of the treated individual is compared to obtain the treatment effects. Individual treatment effects are then aggregated to obtain the total revenue effects for the treated population.

    Limitations

    Measuring total revenue effects is complicated. There are some technical limitations involved in establishing defensible causal connections between our engagements with taxpayers and any improvements to compliance. Even where we establish a causal connection, it can be difficult to understand and identify where our activities resulted in a revenue effect outcome, or an external factor unrelated to us.

    Therefore, we do not yet estimate a wider revenue effect for a number of strategies and products, including:

    • our help services (including information on our website) – used by millions of taxpayers to understand and meet their tax obligations
    • tailored advice, private and public rulings we provide to taxpayers on specific issues (although from time-to-time we can quantify the impact of these interactions for a very small subset of these).

    See also:

      Last modified: 17 Oct 2019QC 53795