103: Evaluation Frameworks That Take Program Diversity Head On

Presenters

Panate Manomaivibool, IIIEE at Lund University
Jensen Montambault, The Nature Conservancy

Abstract

An Evaluation Framework for Policy Learning and Transfer: Extended Producer Responsibility Programs (Panate Manomaivibool, IIIEE at Lund University)

Extended producer responsibility (EPR) has underpinned recycling programs in many Organizations for Economic Cooperation and Development (OECD) countries, particularly in the area of waste electrical and electronic equipment (WEEE). In principle, greater responsibilities to manage end-of-life products should incentivize the producers to make improvements in product and system designs, which in turn, result in better and easier waste management. In practice, the details of EPR programs vary greatly, even in the European Union where a common legal framework exists in the form of the WEEE Directive. This makes it difficult to evaluate the effects of EPR. A framework is developed based on theory-based evaluation and the concept of policy paradigm to delineate between theory and implementation failures. The framework is not only useful for the evaluation of existing programs in OECD countries but has also a potential to deliver policy lessons relevant to the development of new WEEE programs in non-OECD countries. Selected cases from Europe and East Asia are presented to demonstrate the explaining power and discuss the limitations of the framework.

Beyond Carrots and Sticks: A Burgeoning Evaluation Approach to Address Conservation’s Complexity (Jensen Montambault, The Nature Conservancy)

Assessing the impact of conservation has always been inherently complex because of the many confounding factors influencing biodiversity’s response to management. As conservation moves toward integrating socioeconomic impacts, expansive landscape scales, and partnerships with major stakeholders not traditionally associated with conservation (e.g., chemical manufacturers, natural resource extraction companies), the task becomes yet more complicated. At the Nature Conservancy, a suite of tools combining collaboration and accountability has emerged that help us evaluate our programs’ effectiveness in the face of high uncertainty and wide-ranging stakeholder perspectives.  The Nature Conservancy’s approach begins with an “a priori” assessment of the information needed by managers to adapt our conservation work in a dynamic environment. Priority programs participate in internal and external peer-review, receiving written, virtual (internet-based conversations), and in-person feedback supporting careful articulation of expected results, evaluation and response mechanisms. These programs are then held directly accountable to senior managers, including the chief conservation officer and board of directors, on an on-going basis through business plans, management dashboards, and project abstract review. Special resources are assigned for evaluating programs presenting the greatest potential for risk and leverage. This process emphasizes learning and adapting to benefit the organization and wider community of conservation practitioners.

Presentation Materials