104: What’s Complexity Got to Do With It?
Presenters
Jonny Morell, Fulcrum Corporation
Andy Rowe, ARCeconomics
Abstract
Applying “Complexity” to Evaluation: Implications for Models, Metrics, and Methodologies (Jonny Morell, Fulcrum Corporation)
“Complexity” can be understood in three ways: 1) as an intuitive notion of “complicated and hard to understand,” 2) as a mathematical treatment of non-linear dynamics, and 3) as a set of concepts that differ from common sense understanding, but which are useful ways of appreciating how the world works. This session will focus on the third, and is intended to help evaluators develop program theories, metrics, and methodologies that bear on the real world behavior of the programs they evaluate. Some examples of complexity concepts that bear on environmental evaluation include: attractors, power laws, phase shifts, edge of chaos, phase space, emergence, self organization, path dependence, fitness landscape, and co-evolution. This session will be a “think tank” designed to get attendees to think in complexity terms. A few key notions in complexity will be chosen, and scenarios presented (including logic models, metrics, and methodology). Guided discussion will take place in which two evaluations will be sketched, one including the chosen complexity concept, and one not. Comparisons of the value of the designs will then be discussed.
Is Evaluation in Resource, Environmental and Conservation Settings Complex? (Andy Rowe, ARCeconomics)
The premise that evaluation in resource, environmental and conservation (REC) settings is complex underlies this conference as well as much evaluation discourse in conservation, but the evidentiary and logical underpinnings to substantiate the claim is lacking. This paper uses the Simple-Complicated-Complex characterization of Michael Patton to propose that while evaluation in REC settings is rarely simple, it is usually complicated rather than complex. The nature of evaluation in REC settings with a two-system evaluand (human and natural systems) ensures that it is always hard, but rarely complex. The evidence for this argument will draw on evaluations in REC settings conducted by the author over the past twenty-five years. The claim of complexity appears to have more to do with a wish for special methods and treatment and a “science” culture that seeks greater precision for evaluation judgments than feasible or necessary. However existing evaluation approaches are usually sufficient for the job when adapted for the two-system evaluand. Our biggest problem is an extremely weak intellectual infrastructure for evaluation in REC settings.
