Presenting a quality outcome
A quality evaluation of whether, and to what extent, your strategies have achieved your success goals and, ultimately, your desired outcomes, can provide you with a wealth of knowledge on which to base future risk-treatment decisions.
To ensure that you can present a quality evaluation consider the following.
- The analysis methodology - choosing the right methodology is particularly important when looking at attribution issues. You need to select a methodology that can identify whether changes are caused by your strategies or by factors outside your control.
- Any over-emphasis on one or more indicators - while it is acceptable for some indicators to carry more weight in the overall picture than others, you must take care not to rely too much on particular indicators simply because they tell the story that you want them to. Remember: the value you get from evaluating compliance effectiveness is not about success or failure, but rather from the lessons learned and how they feed into the continuous-improvement process.
- Personal bias - you need to resist the temptation to manipulate your evaluation outcomes to paint only a positive picture. Evaluating effectiveness is not meant to be a statement of success or failure - it is meant to provide an understanding of whether you are on the right track with your strategies and to provide some guidance for future planning. Smoothing over any negative results can mask valuable lessons that could influence future risk treatment plans.
- Defensible versus definitive - understanding what the indicators are really telling you can be difficult due to the subjectivity inherent in arriving at a conclusion.
Remember: evaluating your effectiveness is about presenting a defensible story - it is not about providing a definitive measurement. Knowing where to draw the line between definitive and defensible is quite difficult because your level of comfort in drawing conclusions about effectiveness is intrinsically linked to your personal risk stance.
When deciding on the level of information, you need to support your story of effectiveness, thinking about how comfortable you would be if you were asked to justify your conclusions. Involving others with relevant business knowledge in the evaluation can go a long way to increasing your comfort levels.