Literature review: measuring compliance effectiveness 2007

4.0 Measurement methodology

There has been a global trend towards evidence-based government administration in the past three decades (Treasury Board of Canada 2005, page 1).

The most important criterion for good governance and public management may well be that decision-making processes must take into account evidence of program effectiveness. Although such evidence is rarely definitive, without any evidence resources will be allocated to programs based merely on preferences (Treasury Board of Canada 2005).

The Treasury Board of Canada has written about its government's lack of demand for evidence, and use of it, for drawing conclusions about performance effectiveness. It poses the question: Will the recommended [evaluation] method provide adequate evidence in relation to the issues of concern, on time and within budget? (1998, page 32). This has implications for organising information collection and analysis to ascertain whether goals have been met against each effectiveness indicator.

This section discusses collecting and analysing evidence to evaluate whether indicators have been met. It does not address other processes involved in measuring compliance effectiveness, such as formulating, developing or reviewing indicators; providing guidance and support to stakeholders involved in undertaking measurement; or feeding results into decision making. These topics are discussed in sections 3 and 5 of this review.

Sections within 4.0 Measurement methodology

• 4.1 Attribution and causal inference
• 4.2 Direct and indirect effects
• 4.3 Qualitative versus quantitative data
• 4.4 Information collection and analysis
• 4.5 Cost-benefit of collecting data
• 4.6 Data integrity issues
• 4.7 Data systems