4.1 How will the evaluation be conducted?

Use the following measurement methods to determine which are most relevant to your identified indicators.

  • Baseline comparisons - These compare your actual performance after the compliance strategy to performance before the strategy. Not all changes in the indicator may be attributable to the strategy - however, without a baseline it will be difficult to assess whether the strategy has made any difference at all.

Example: The lodgment figures for 2005-06 income tax returns (60.6% lodged on time and 69.2% lodged by the end of the income year) could be used as baseline figures.

End of example
  • Benchmarks - Benchmarks are comparisons to other areas that have similar characteristics. In achieving the benchmarks, distinguish the impact of the compliance strategy from other factors.

Example: A benchmark for lodging company tax returns on time in Australia might be the proportion of on-time lodgment for Australian Securities & Investments Commission (ASIC) annual returns or the proportion of on-time lodgments achieved in the United States or the United Kingdom for similar populations.

End of example
  • Randomised controlled trials - These trials use a pre-selected randomised group (control group) that has identical characteristics to the target group. The control group is isolated from the target group, with both groups measured before and after treatment. Using a control group minimises the impact of other factors in measuring effectiveness. However, it is not always possible to identify control groups - for example, if there is an Australia-wide marketing strategy using mainstream media, most people will be exposed to the strategy.

Example: A group of taxpayers are randomly selected to test the effectiveness of an advisory notice on their compliance behaviour. Separating the group into a control group and a target group, then sending the advisory notices only to the target group, allows for differences in behaviour between the groups to be used as an indicator for the effectiveness of the strategy.

End of example
  • Comparison group studies - While randomised controlled trials use groups with identical characteristics, comparison group studies use groups with similar characteristics. The comparison group which does not receive the treatment is compared to the target group, to find the difference in behaviour. Results from studies using well-matched groups can approach the rigour of a randomised controlled trial.

Example: A pilot marketing strategy is undertaken in Brisbane, and the recognition levels in Brisbane after the strategy are compared to those in Adelaide where the pilot was not run.

End of example
  • Standards - Standards are pre-defined, minimum acceptable levels of performance, including service and quality standards. A standard may be chosen from ideals, developed statistically, or by comparison with similar organisations. They are often used in conjunction with baselines, with the assumption being that achieving the standard is due to the compliance strategy that has been implemented, and not to other factors.

Example: A quantitative standard for on-time lodgments could be represented by a minimum of 75% of returns lodged on time.

End of example
  • Targets - Targets are quantifiable performance levels set for a date in the future. Targets need to be realistic and achievable, otherwise they will be ignored. The reasons for choosing the targets and how they were derived must be documented in the evaluation plan.

Example: A 10% increase in the uptake of individuals who prepare and lodge their own returns using e-tax (electronic preparation and lodgment) for the 2007-08 income year.

End of example
  • Trends over time - Trends provide comparisons with earlier time periods and can show whether or not a sustained impact over time (usually six or more years) has been achieved. As other factors may affect trends, look for control groups or benchmarks to help interpret the trend.

Example: Use the four years prior to the 2007 year tax return baseline figures, track changes over 2008 and 2009, and continue to monitor trends in 2010.

End of example
  • Based on these methods, determine what needs to be done to establish your evaluation approach and monitor the indicators selected.
Last modified: 19 Jan 2015QC 21200