Is an upgraded methodology to assess the impacts of different conservation instruments in order? Jesus Perez., Creative Commons.

Scarce conservation funding is currently under public scrutiny. “Does it work?” is frequently the overarching question – and if so, is this impact quantifiable?

We have thus seen various impact evaluations studies popping up with single conservation bottom lines (i.e. from year A to B, the intervention X saved Y hectares of habitat or forest carbon). But these isolated results are hardly satisfactory.

For this very reason, a few years ago, scientists Daniela Miteva, Subhrendu Pattanayak and Paul Ferraro called for a scientific upgrade- what they call Conservation Impact Evaluation 2.0. Their new approach sought to answer the following questions: How do quantitative impacts of different policy tools compare? How do impacts contextually vary, producing heterogeneous effects inside their target areas? Can we account for spillover effects towards adjacent, non-targeted areas? What are the simultaneous effects on local livelihoods? And, by the end of the day, how cost-effective are these interventions?

All these questions, and more, are now being answered in a comprehensive and innovative new article authored by Katharine Sims from Amherst College and Jennifer Alix-Garcia from the University of Wisconsin-Madison. In the article, which was published in the Journal of Environmental Economics and Management, the authors juxtaposed conservation sticks and carrots in Mexico.

They compared forest conservation effects (changes in forest cover from 2000 to 2012) and poverty indicators (2000-2010) of Protected Areas (PA) to Payments for Environmental Services (PES) in Mexico over the past decade. They used “locality” as the spatial unit of analysis, and the degree of treatment is measured continuously as the share of land under PA and PES respectively, thus accounting for the possible leakage of pressures towards adjacent, non-targeted lands. Mexican protected areas are subdivided into strict PA, mixed-use PA, and biosphere reserves (see map below).

For comparison purposes, treatment localities were “matched” with localities facing similar land-use history and deforestation risks in the same federal state to avoid comparing apples with oranges.

Locations of PES and PAs in Mexico by 2010

ENVIRONMENTALLY EFFECTIVE?

What were their key results? The good news is that both policies worked for the environment by avoiding 20-25 percent of expected forest loss, and neither harmed local livelihoods (see graph below).

Firstly, PA protection effects were notable, though also variable across sub-categories. Impacts were particularly impressive for the biosphere reserves, which combine strictly-protected core zones with mixed-use buffer zones. Previous studies had found that Mexico’s PAs were not slowing land-cover change, but this study finds that in the most recent decade they avoided substantial deforestation, possibly because PA funding increased markedly.

Secondly, PES also proved environmentally-effective. While PES currently cover a much smaller proportion of land area than the PA combined, the percentage change impact on deforestation was similar.

With respect to human development impacts, the authors used a poverty alleviation index based on indicators such as education, housing, sanitation, infrastructure, and household assets. Communities with higher proportions of land in both PES and PA alleviated local poverty over time, demonstrating that all land conservation measures met an absolute standard of “do no harm”.

The relative impact assessments, which compared these changes over time to the appropriate counterfactual case, suggest that biosphere reserves and mixed-use PAs had neutral poverty impacts, while strict PAs did not alleviate poverty as quickly as comparison areas (as indicated by the negative score in the figure below). PES, however, did significantly alleviate poverty (by about 10-12 percent), and consistently so across sub-indicators.

Estimated impacts of conservation interventions (percent change) on forest cover and poverty alleviation. Positive bars indicate gains for conservation and livelihoods. Whiskers indicate 95 percent confidence intervals.

Overall, the study results corroborate some of the expected tradeoffs between these policy types. Well-funded and enforced PA, such as the biosphere reserves, are more likely to achieve complete environmental protection, while PES are more likely to directly alleviate poverty.

COST EFFECTIVENESS

But what about cost effectiveness? The authors looked at the opportunity costs of land conserved, which arguably represents the largest social cost (although implementation costs may still be key to the choices of budget-holding agencies). Perhaps here the results are more surprising.

Environmental economists would often expect incentive-based policies to be of lower cost than command-and-control regulation, since service providers self-select the land areas they offer for participation (the rational landowner will, at a given price, choose the ones with the lowest cost).

Yet, the areas where PA and PES are located in Mexico both show a mix of land exhibiting high and low opportunity cost. This probably reflects the efforts of the Mexican Forestry Commission to prioritize applying PES to areas of high threat or opportunity cost. In addition, areas where the PAs are located might once have been “high and far” areas of low threat, but are under increasing pressure in this decade.

In conclusion, this study definitely exhibits some upgraded impact evaluation features that deserve to be replicated in other case studies. As countries develop strategies for climate change mitigation, including REDD+ and/or biodiversity protection, a comparative evaluation of how much bang for the conservation buck different conservation instruments have delivered seems quintessential.

This would require first putting into perspective what we consider a “conservation success”. For instance, the systematic PES review by Cyrus Samii & Colleagues (2014), although eventually based on only nine rigorous case studies in two countries (Costa Rica and Mexico), ended up concluding that PES were relatively ineffective in protecting forests: “The vast majority of parcels on which payments are issued are made ‘for nothing’ from an ex post perspective … so that the PES effect is modest and seems to come with high levels of inefficiency.”

Yet, in a recent PLOS ONE collection of new forest conservation impacts studies, scientists including myself compared the impacts across different policy instruments and found that PES effectiveness actually fared quite well.

The new article from Sims and Alix-Garcia now goes an important step further to broaden our perspective by comparing conservation impacts for different instruments in the same site and using identical methodology. This upgraded methodology should enable us to improve the design of policy mixes where different conservation instruments are used to complement each other.