Elements of a systematic review, adapted from Petrokofsky et al. 2011 and reproduced by kind permission of the Commonwealth Forestry Association

There are increasing expectations that policies should be evidence based. This extends to the use of public funds for development assistance, for example Government initiatives such as those promoted by national Governments and supported by initiatives such as those of the Overseas Development Institute’s RAPID programme, Research to Action and 3ie.

While recognizing that scientific research is an important source of evidence, it is important to clarify that science does not per se provide answers to policy issues, nor is it the only source of what counts as authoritative evidence (1). Some of this confusion is reflected in the way that evidence-based approaches have sometimes been criticized for usurping other forms of evidence and for suggesting technological solutions to complex socio-economic problems. These same critics, however, are also arguing for effective decision-making that is based on sound evidence. For example, in international climate change policy, huge efforts have been made to clarify the knowledge base and inform decision-makers of available evidence, but those same decision-makers remain largely indecisive. This appears to depend more on the decision-making process than on the quality and volume of evidence provided. There is much more going on here: it is not a simple linear relationship between good science and good policy.

In a significant policy paper, such as the recent internal evaluation of World Bank forestry operations (2), it is important that its analysis is systematic, transparent and inclusive of available information, as discussed in this CIFOR blog entry. Otherwise, the paper, and subsequent commentaries and decisions, can quickly be considered biased or ignorant, or both (3).

The evidence process is iterative. It entails defining the policy questions and subsequently reviewing the state of knowledge that is available to answer this question. This can lead to new priorities in research, as well as a re-examination of the initial policy framework. In addition, new results can be added to the body of knowledge, potentially improving the quality of conclusions.

It is worth recalling J. Nisbet’s words from over a century ago: “Forestry… may be called the younger sister and the servant of Agriculture. Each of these arts is essential to the welfare of nations, and no people can be said to be wise, politic, or economic which does not pay attention to the advancement of both.” (1)

The value of forestry in shaping landscapes is as important now as it was when Nisbet was working as a professional forester. However, he would have been surprised by the dramatic shifts in the services and products that woodlands of the 21st century are expected to deliver. And there are new challenges as well, with rapid environmental change, different pests and diseases, and novel economic and social conditions to consider.

Many of these changing directions and uncertainties require new forest policies and research to guide decisions and achieve beneficial outcomes for society. This places heavy demands on science to provide the information to form sound, evidence-based policy. However, it is not necessary always to commission new research to provide answers to new policy questions or new ways of looking at the world’s most serious problems. The existing knowledge base contains a huge volume of potentially highly relevant research, much of it of very high quality. The challenge is to mine it and analyse the findings in ways that support unbiased policy making.

One current re-examination of a policy framework is the broadening of perspectives on forestry and agriculture leading towards a landscape approach, to pursue better and combined solutions in meeting food security, climate change, nature conservation and poverty challenges. To inform existing institutional and policy arrangements about the benefits of moving to a landscape approach, a solid evidence base is key. Otherwise, there is a risk that (in this case)  sections of the forestry community may continue to focus on isolated issues like timber concessions and protected areas, since these are easily defined and institutionally convenient to deal with.

With these priorities in mind, CIFOR made a strategic decision to introduce an evidence-based forestry initiative as an important component of our work in 2013. Currently in the initial phases of this initiative, CIFOR is joined by the University of Oxford and key partner organizations to develop a programme that will put robust evidence at the heart of our work.

A significant part of the role of CIFOR and its partner organizations in international forestry research is to provide evidence for policy processes, whether directly through defined research projects or indirectly through policy-makers and other stakeholders using our published outputs. These outputs can be our peer-reviewed research papers or they may be informal publications, such as our irregular statements made through social media channels (Twitter, blogs, etc.). In all cases, it is fundamental to have high standards and well-defined approaches to assessing the quality of the evidence.

Many sectors have well-established procedures for using systematic, evidence-based approaches to science–policy dialogue and action. These include health care (Cochrane Collaboration, which is celebrating its 20th anniversary in 2013), social welfare (Campbell Collaboration, which has been active since 2000), and environmental conservation (Collaboration for Environmental Evidence, which started operating in 2006).

Systematic reviews are at the heart of the evidence-based initiatives in these fields. They are powerful information tools that aid good decision-making: without systematic reviews there is no evidence-based medicine. But systematic reviews are not yet a common practise in the forestry community, despite the fact that the demand for decision-making to be based on evidence continues to grow (4). Terms like ‘evidence-based policy’ are used freely in international and national policy briefings in forestry, largely without definition (5).

The sheer volume of forest science published in the international literature, and the cost and difficulty of reading through it to gain understanding of complex, often conflicting, messages are daunting for policy-makers. As Roger Pielke wryly observed in his influential book The Honest Broker:

“The scientific enterprise is diverse enough to offer information that can be used to support a diversity of perspectives on just about any subject… deciding a course of action and then finding information to support it is common across the political spectrum”

Systematic reviews were originally developed to stop this ‘cherry-picking’ of scientific (and other) evidence in medicine. They are designed to handle large numbers of studies without introducing additional biases.

CIFOR aims to draw on the experiences and expertise of these other fields to scope the potential for establishing a similar ‘collaboration without walls,’ producing systematic reviews as a core activity. We are excited to be working with our key partners on this initiative and to start a collaborative process of improving the quality of evidence used to make decisions for some of the most pressing problems that challenge sustainable development in the 21st century.

I would like to express my gratitude to Gill Petrokofsky for her invaluable contribution and inputs to this blog.

For more information on the evidence-based forestry initiative, please contact Gill Petrokofsky or Peter Kanowski.

Bibliography

1. Sutherland W.J, Bellingan L, Bellingham J.R, Blackstock J.J, Bloomfield RM, et al., 2012. A collaboratively-derived science-policy research agenda. PLoS ONE 7(3): e31824. doi:10.1371/journal.pone.0031824

2. World Bank, Independent Evaluation Group, 2013. Managing Forest Resources for Sustainable Development. An Evaluation of World Bank Group Experience, February 5, 2013

3. Nisbet J., 1905. The Forester. [2 Volumes] UK: William Blackwood and Sons.

4. European Commission, 2010. Evidence-based policy versus policy-biased evidence: the challenge of feeding scientific advice into policy-making. Conclusions of the High-Level Trans-Atlantic Science for Policy Workshop, October 2009, Ispra, Italy.

5. Petrokofsky G., Holmgren P., Brown N.D., 2011. Reliable forest carbon monitoring: systematic reviews as a tool for validating the knowledge base. International Forestry Review 13(1):56-66.