• When should an evaluation occur?

    Identifying when an evaluation should occur is based on when you expect to see an effect.

    In deciding when to undertake an evaluation, consider the following questions:

    • When do you expect to start seeing the change?
    • When do you think the full effects will be seen?
    • How often will you need to do an evaluation to be able to understand whether the change has been sustained?

    Figure 1: Compliance effectiveness methodology

    phase_1 Articulate risk

    phase_2 Define outcomes

    Checkpoints

    You should ensure that:

    • the risk aligns with our intent
    • the risk adequately reflects the behaviours and drivers
    • the risk is refined, concise and not open to interpretation.

     

    Checkpoints

    You should ensure that:

    • the desired outcomes align with our intent and the risk you have identified
    • the overall desired outcomes are adequately covered by your specific (success) goals
    • the strategies target participants of the risk and drivers of the behaviour
    • the strategies address any unintended consequences.
     

    phase_3 Design indicators

    phase_4 Validate indicators

    Checkpoints

    You should ensure that:

    • each success goal can be measured and has a corresponding indicator (or suite of indicators)
    • potential indicators are aligned with the success goals, desired outcomes and our intent
    • indicators are viable and provide a balanced picture of performance
    • both qualitative and quantitative information is used.

     

    Checkpoints

    You should ensure that:

    • data exists, is available, or can be acquired at a reasonable cost
    • we can explain how effective our compliance strategies have been - that is, the extent to which actual outcomes align with desired outcomes.

     

      Last modified: 13 Jan 2015QC 25789