2009 EEN Forum
History
The fourth EEN Forum was co-hosted by the U.S. Environmental Protection Agency (EPA) and the National Fish and Wildlife Foundation (NFWF) and the George Washington University Trachtenberg School of Public Policy in Washington, DC on June 8 and 9, 2009. More than 230 participants were in attendance from federal government agencies, foundations, private evaluation firms, academia, non-profits and other organizations.
Themes
The theme of the fourth EEN Forum was “The Value of Environmental Evaluation.” Over two days participants discussed the role that environmental evaluation plays in their organizations including successes, pitfalls and recommendations for moving toward an Era of Effectiveness.
Concurrent sessions covered three areas:
1. Using Evaluation in the Early Stages of Environmental Programs and Policies
2. Using Evaluation to Improve Mature Environmental Programs and Policies
3. Building the Capacity and Culture of Evaluation
Forum Documents
Detailed Agenda – EEN 2009 Forum
This is the EEN 2009 Forum – Detailed Agenda for the forum held June 2009 in Washington, D.C.
Participant List – 2009 EEN Forum
This is a complete list of participants and biographies for the 2009 Environmental Evaluators Forum.
Presenters Biographies – 2009 EEN Forum
This is a complete list of presenters and biographies for the 2009 Environmental Evaluators Forum.
Keynote Presentations
06/08/2009 – Keynote Presentation 1
Presenters: Marcia Mulkey (US EPA), Rowan Gould (Fish and Wildlife Service), Mark Shaffer (Doris Duke Charitable Foundation), Debra Rog (Westat)
06/09/2009 – Keynote Presentation 2
Presenters: Mikael Hildén (Finnish Environment Institute), George Grob (Center for Public Program Evaluation)
Plenary Sessions
06/08/2009 – Economic Recovery and Environmental Goals: Opportunities for Evaluation • Session Notes
Description: Evaluators and evaluation consumers give perspectives on the value of evaluation in addressing new priorities set by new leadership during a global economic crisis.
06/09/2009 – Forum Synthesis and Next Steps for the EEN
Description: Panelists reflect on next steps for the EEN, how the network supports and advances evaluation, and the future role of environmental evaluation.
06/09/2009 – Creating Common Standards in the Environmental Community: Learning from Nonprofits • Session Notes
Description: Panelists describe the Conservation Measures Partnership and the open standards for the practice of conservation that all CMP members commit to. Discussion focuses on the opportunities and challenges associated with adopting and implementing open standards in the environmental community.
Evaluation in the Early Stages
06/08/2009 – Evaluation and Conservation Planning
Presentors: Andrew Knight, Devra Kleiman, David Callahan
Description: These case studies highlight the challenges of conservation planning and will be used to share knowledge, examine the gap between “knowing” and “doing” effective conservation planning, and to focus group discussion towards identifying avenues for improving the effectiveness of evaluation for conservation planning initiatives.
06/08/2009 – Case Study of NFWF Keystone Initiatives
Presenters: Timothy Male, Tony Chatwin, Christina Kakoyannis
Description: To better demonstrate the environmental or social impacts of the organization’s grantmaking investments, the National Fish and Wildlife Foundation provides additional evaluation staff and leadership support to embed a more comprehensive system of monitoring and evaluation into the entire lifecycle of the Foundation’s new long-term initiatives.
06/08/2009 – Case Study of Moore Foundation’s Amazon Conservation Initiative
Presenters: Luis Solérzano, Jared Hardner
Description: This presentation provides an overview of planned conservation outcomes for the Andes Amazon Basin. Presenters explain how a new plan enables easier internal and external evaluation and attempts to overcome two of the greater evaluation challenges that face conservation programs–counterfactuals and attribution.
06/08/2009 – Experimental and Quasi-Experimental Evaluation Designs: Perspectives From Academia
Presenter: Paul Ferraro
Description: Three case studies are used as vehicles to emphasize that experimental and quasi-experimental designs are feasible, useful and comprehensible to a wide audience.
06/08/2009 – Complex Evaluation Methods – The Evaluator as Objective Analyst and Salesperson (and Occasional Punching Bag)
Presenter: Lou Nadeau
Description: How do we get program managers to buy in to using more complex (and hopefully more valid) designs? What happens when the use of more sophisticated methods shows that a program is ineffective? This presentation will look at the roles of the evaluator that uses sophisticated methods.
06/08/2009 – Application of Criteria and Indicators for Forest Sustainability at the Local Government Level • Session Notes
Presenter: Don Outen
Description: This presentation explains how Baltimore County is applying the Montreal Process Criteria and Indicators for forest sustainability. Working through a multi-party Steering Committee and several special stakeholder Forums, the County structured management programs under the premise that better data leads to better dialogue and decisions.
06/08/2009 – Environmental Evaluation Practices and the Issue of Scale • Session Notes
Presenter: Hans Bruyninckx
Description: This presentation examines the problem of scale. Social scientists, natural scientists, and evaluators have not properly defined the concept of scale for environmental problems. Environmental scale generally differs from social scale, which confounds the challenge of evaluating policies and governance arrangements in addressing environmental issues.
06/08/2009 – An Early Evaluation of NOAA’s Habitat Matrix Program • Session Notes
Presenter: Bruce McDowell
Description: The presentation details a report that makes recommendations for establishing an overarching statutory framework to unify NOAA’s programs around outcome oriented goals, strengthening the performance goals and measures being developed and the related annual targets, increasing scientific support for habitat assessments and progress reporting, and working more closely with the program’s numerous partners and stakeholders.
06/08/2009 – The Value of Process Evaluation: Risk Reduction Measures for Pesticide Products Could Be Implemented up to Four Years Sooner • Session Notes
Presenters: Debra Kemp, Peter Caulkins
Description: EPA sponsored an external review of the Office of Pesticide Programs to identify opportunities for streamlining the product reregistration process. The evaluation team provided EPA with 25 recommendations that covered all phases of the process, as well as communication, performance management, information management, resources, and staffing. The evaluation led OPP to make significant changes in its program.
06/09/2009 – Scale and Scope in the Application of Criteria and Indicators for Forest Sustainability at the National Level • Session Notes
Presenter: Guy Robertson
Description: The Forest Service recently released its Draft 2010 National Report on Sustainable Forests which relies on the Montreal Process Criteria and Indicators for Forest Sustainability but, at the same time, it is guided by a broad public collaboration process. This presentation gives specific examples of how this balancing act has lead to compromises in terms of both the scale and the scope of data presented in the report.
Evaluation in the Mature Stages
06/08/2009 – The Chesapeake Bay Small Watershed Grants Program
Presenters: Amanda Bassow, Laura Carrier, Andy Rowe
Description: The Chesapeake Bay Small Watershed Grants Program has been awarding grants for local watershed stewardship and restoration projects since 1999. In 2007, GHK Int’l completed a third-party evaluation of the program. This session examines the changes that have resulted from the evaluation, and how those changes have been received by grantees.
06/08/2009 – A Case Study in How Evaluation Brought Change to the Environment Enforcement Training Programs at US EPA • Session Notes
Presenters: Zena Aldridge, Yvonne Watson
Description: When the EPA’s National Enforcement Training Institute (NETI) suggested several changes to improve the environmental enforcement training program in May 2006, senior managers could not come to consensus. Using information gained from a program evaluation project completed one year later, however, NETI recommended similar changes once again and senior managers came to a different conclusion: substantial changes in the program were approved and implemented. This presentation will highlight key components to this turnaround.
06/08/2009 – Evaluations and Environmental Education: Challenges and Successes
Presenters: Kathleen MacKinnon, Kathy McGlauflin, Judy Braus, Connie Kubo Della-Piana, Leslie Goodyear
Description: Four case studies are presented exploring the challenges of evaluating environmental education programs.
06/08/2009 – Recognizing the Value of Evaluation: Fostering Positive Stakeholder Interactions for Environmental Management
Presenters: Ken Genskow, Tom Koontz, Suzi Ruhl
Description: While many evaluation efforts come from evaluation professionals, it is important to consider the role that stakeholders can play in evaluation. This panel provides diverse perspectives about how to foster positive stakeholder interactions.
06/08/2009 – Data and Information: Access, Use and Finding Meaning • Session Notes
Presenter: Ed Washburn
Description: When you’re making a good faith effort to demonstrate outcomes and beneficial impacts that are attributed to your environmental program — and your program budget keeps shrinking – and the ‘transparency lamp’ shining on your program keeps getting brighter, where do you turn for evaluation data? The GEOSS (Global Earth Observation System of Systems) might be a faint ray of hope.
06/08/2009 – Data and Information: Lessons Learned Through the State of the Nation’s Ecosystem Project • Session Notes
Presenter: Anne Marsh
Description: This presentation describes two State of the Nation’s Ecosystems reports (2002 and 2008) created to populate stakeholder-designed indicators on coasts and oceans, farmlands, forests, fresh waters, grasslands and shrublands, urban and suburban systems and the nation as a whole.
06/08/2009 – Using Evaluation Methods as Decision Support for Strategy Development • Session Notes
Presenter: Richard Gelb
Description: This presentation describes how tools and methods historically used in the environmental evaluation realm can be applied throughout the performance and adaptive management cycle as decision-support to a variety of actors, with a particular focus on the role of intermediate outcomes and geospatial analysis.
06/08/2009 – Building a Network’s Capacity for Evaluation • Session Notes
Presenters: Jennifer Splansky, Leslie Goodyear
Description: Three case studies describe the framework for evaluation capacity building among NGOs and non-profit foundations.
Building Evaluation Capacity and Culture
06/09/2009 – Repositories of Projects and Evaluations: A Foundation for Evidence-Based Decisions • Session Notes
Presenter: Andrew Pullin
Description: This talk considers the value and necessary characteristics of a repository of evidence to support decision making in environmental management.
06/09/2009 – The Conservation Registry: Promoting Strategic Conservation and Sharing Knowledge • Session Notes
Presenter: Gina LaRocco
Description: The transformative effect of Internet technologies is reshaping how conservation happens. Defenders of Wildlife and many partners have developed an online tool, the Conservation Registry, that allows all types of users, including agencies, organizations, private landowners, and policy makers, to record, track and map conservation projects occurring across the landscape.
06/09/2009 – Effective Communication of Evaluation Results and Learning
Presenters: Per Mickwitz, Shelley Robertson
Description: The presentation discusses dissemination and production based strategies for communication of evaluation results. The presentation uses experiences from the development of eco-efficiency indicators in the Kymenlaakso region of Finland. The second half of the session, continues the theme by identifying ways to communicate results and encourage learning.
Presenters: Lisa Flowers, Annelise Carleton-Hug
Description: This session explains the fundamentals of utilization-focused program evaluation: evaluations designed to provide data that is useful for program managers and other interested stakeholders.
06/09/2009 – Competency, Capacity and Culture: Toolboxes for Evaluators • Session Notes
Presenter: Michaela Zint
Decription: Many environmental professionals are interested in increasing their evaluation competency and yet, little is known about the extent to which various experiences may help them achieve this outcome. This presentation explains an online tool called “My EE Evaluation Resource Assistant,” or MEERA.
06/09/2009 – The Handbook of Environmental Policy Evaluation • Session Notes
Presenter: Ann Crabbé
Description: The Handbook of Environmental Policy Evaluation is a guide to environmental policy evaluation in practice. Beginning with an introduction to the general principles of evaluation, it explains the particular complexities native to the environmental sphere and provides a comprehensive toolkit of evaluation methods and techniques which the practitioner can employ and refer to.
06/09/2009 – A Tool-Kit for Agricultural Research Impact Evaluators: Report on a Work in Progress • Session Notes
Presenter: Doug Horton
Description: This presentation describes a Tool-Kit for Impact Evaluation. The Tool-Kit under development will provide an overview of impact evaluation, the different tasks involved, and the different tools and approaches that can be used for each task.
06/09/2009 – Program and Policy Evaluation in State, Local and Tribal Governments
Presenters: Mary Beth Brown, Warren Kimball, Jennifer Falck
Description: In this session, the presenters discuss the value of evaluation to decision makers and citizens in a state, local or tribal environment. The presenters will pose the question, “When does data become information?”
06/09/2009 – Working Session Discussion: EEN Products and Services • Session Notes
Description: This section includes the Notes on the EEN Products Working Session and the Evaluation Memo of the 2009 EEN Forum.
This section will serve as a clearing-house of documents and information related to the network.
