What’s Complexity Got to Do With It?
Evaluation in the Face of Uncertainty: Maximizing Methodological Choice When Unpredictable Outcomes are Likely
Jonny Morell, Fulcrum Corporation
Unintended consequences of program behavior can be arrayed on a continuum from “those which could have reasonably been foreseen” (at least in dim outline), to those which cannot be predicted because they emanate from complex system behavior. Different evaluation tactics are relevant along this continuum. This presentation will begin with an explanation of what makes systems complex, and proceed to a brief overview of evaluation tactics that are useful toward the “unpredictable” end of the continuum. It will conclude with a framework for making methodological choices, given the fact that any tactic implemented may induce new difficulties for conducting an successful evaluation.
Is Evaluation in Resource, Environmental and Conservation Settings Complex?
Andy Rowe, ARCeconomics
The premise that evaluation in resource, environmental and conservation (REC) settings is complex underlies this conference as well as much evaluation discourse in conservation, but the evidentiary and logical underpinnings to substantiate the claim is lacking. This paper uses the Simple-Complicated-Complex characterization of Michael Patton to propose that while evaluation in REC settings is rarely simple, it is usually complicated rather than complex. The nature of evaluation in REC settings with a two-system evaluand (human and natural systems) ensures that it is always hard, but rarely complex. The evidence for this argument will draw on evaluations in REC settings conducted by the author over the past twenty-five years. The claim of complexity appears to have more to do with a wish for special methods and treatment and a “science” culture that seeks greater precision for evaluation judgments than feasible or necessary. However existing evaluation approaches are usually sufficient for the job when adapted for the two-system evaluand. Our biggest problem is an extremely weak intellectual infrastructure for evaluation in REC settings.
