Indicators should be able to measure the expected or desired changes. They should also have a direct link with your success goals and desired outcomes.
Don't just choose an indicator simply because it's easy to measure - make sure it contributes to the evidence you're gathering to help you understand the extent to which your success goals have been achieved.
Individual taxpayers voluntarily comply with their income tax lodgment obligations.
A change in the percentage of individual returns lodged by the due date.
An increase in this indicator will provide evidence of improved voluntary compliance with lodgment obligations.
You also need to understand what the indicator should show in the immediate and longer term. For example, a successful strategy may show a significant improvement in the short term that levels off in the future - an immediate effect with a sustained change.
End of example
You need to understand whether your strategies affect voluntary compliance and community confidence over time. Select indicators that will identify change and show your progress over the immediate, intermediate and long term.
Indicators also need to be based on data that can be produced regularly enough to track progress and quickly enough for it to be useful, with only a short time between the period the data covers and when it becomes available. Consider whether:
- there is a time lag before the data becomes available
- the data is available on a regular or infrequent basis
- the data is available for all of the relevant time period.
There is often a time lag between the collection and availability of external data. You need to carefully examine your data sources to understand how timing issues will affect the usability of your indicator.
Change in the number of individual income tax returns lodged compared to the number of people identified in the Australian Bureau of Statistics' (ABS) Australian labour force statistics.
These statistics are generally presented as data which spans a calendar year rather than an income year. There is a notable time lag between collection and presentation.
When comparing this ABS data against our data, ensure the correct months are represented in the data set for the income year, rather than calendar year, and that they correspond to the year in which lodgment behaviour is being assessed.
End of example
Additional validation tests
When validating your indicators, you should also make sure that they are:
- Attributable - indicators need to measure something that your strategies can reasonably be expected to influence.
You need to think about whether a change in behaviour is the result of your strategies, or whether that change was influenced by some other factors.
Bear in mind that:
- the behavioural change might have occurred regardless of your compliance strategies
- your strategies may only be partially responsible for the observed change because of some other influencing factor
- there may be no obvious change in behaviour because your strategies have halted deterioration or maintained the status quo.
Where other factors are likely to have an impact on the usability of the indicator, record both the issue and how you expect it to affect the results.
Other factors that could cause a change in behaviour include:
- our other strategies
- court decisions on tax matters
- changes in tax rates
- economic fluctuations
- natural disasters
- media coverage of tax related issues
- taxpayer literacy and numeracy levels.
Find out more
For more information about external factors, refer to pages 46-48 of our Literature Review - Measuring compliance effectiveness (NAT 71078).
End of find out more
- Comparable - continuity is necessary in order to make comparisons in trend analysis over time.
- Data can be subject to continual revision - for example, new data may be received over time that may have an impact on your original benchmark population. While it is not always possible to avoid differences in the underlying data, recognition of those differences and the influence they have on the indicator analysis should be documented.
- Able to be substantiated - the results and conclusions drawn during the analysis of an indicator should be capable of being substantiated by an independent authority. In other words, an independent reviewer with relevant qualifications should be able to come to the same conclusions.
- Unbiased - the information used to measure effectiveness should be able to be impartially collected, analysed and evaluated.
- For example, when data is reported can result in a bias - taxpayers who are due a refund tend to lodge earlier than those who will end up having a tax debt. The timing of the collection and analysis may produce different results.
In the spirit of continuous improvement, it is essential to not only embrace our successes but to learn from our experiences. The results must present a true picture of the extent to which we have achieved our success goals.