How to evaluate climate policy: case of an evaluation-based performance audit of Finnish climate and energy strategy

Paula Kivimaa, Finnish Environment Institute

Evaluations of climate policy are challenging both prior and after implementation of new policies. They are compli­cated by uncertain cause-effect chains, multiple other causes and policies influencing in desired and opposite direc­tions, slowly changing practices, and long time periods before outcomes can be measured. Compared to some other environmental policy evaluations, climate policy evalua­tions are more challenging as their design and implemen­tation is often a result of several administra­tive sectors. Climate policy typically requires coherence between, for example, energy, transport, forest and agricultural policies. Previous evaluations can point out possible methods to carry out climate policy evaluations but also areas in need of further improvement. This presentation provides an example through an evaluation-based performance audit of the preparation and implementation of Finnish Climate and Energy Strategy.

Performance audits, carried out by supreme audit institutions, typically assess the efficiency and effectiveness of government organisations, activities, and policies by investigating out­puts and outcomes of organisations, activities, and policy programmes. They are similar to evaluations and use many common methodologies. Between August 2010 and October 2011, an audit of the Finnish Climate and Energy Strategy, issued in 2008, was carried out by the National audit office of Finland. The implementation part of the audit focused on the efforts for consistency, effec­tiveness and cost-efficiency from the perspective of climate change mitigation. In particular, evaluating the effective­ness was challenging, because the outcomes of policies on emissions could not be detected after a short time period. Thus, the outcomes were evaluated tentatively based on the identified outputs. The audit used a variety of methods and data sources to derive conclusions on the level of implemen­tation and out­comes, including a stakeholder questionnaire, analysis of government budget appro­priations, expert interviews, policy documentation and statistics. It pointed out that a climate policy evaluation based on multiple quantitative and qualitative data sources can not only trace the outputs but also produce a provisional evaluation of effectiveness.