For projects to combat land degradation and desertification

The French Scientific Committee on Desertification (CSFD), in association with civil society, has developed a set of indicators allowing decision makers and development actors to evaluate the local impacts of operations to combat land degradation and desertification.

This study is in a dual context of the United Nations Convention to Combat Desertification and the desire to evaluate actions outside France funded by the French Development Agency and the French Ministry of Foreign Affairs.

Why evaluate and for whom?

Evaluating the impacts of actions and projects to combat land degradation and desertification and projects for sustainable land management is essential in order to:

• Check the relevance and coherence of the initial goals.
• Shed light on how these actions are conducted and managed, enhance their effectiveness and relevance, reorient them if necessary and ensure sustainability by capitalizing on the assets and accountability of the actors.
• Report on the implementation, results and impact of action to combat desertification beyond its direct stakeholders.
• Study the sustainability of the observed effects (impact).
• Document learning and capitalisation processes, information campaigns, communication or advocacy, especially for donors, by formulating economic arguments—or others—to encourage investment in actions to combat desertification.
• Explain / analyse a complex reality and how it operates.

It is also a democratic requirement in regard to the citizens and parliaments who finance them. Assessing the impacts of projects to combat desertification is thus necessary:

• For civil society, as the evaluation process has a formative effect on local stakeholders combating desertification (collective learning system) and enables local actors to become aware of the value of natural resource management.
• For national policies as an instrument for mobilizing resources to combat desertification. This means transforming scientific results into arguments for these policy makers.
• For international decision makers to encourage them to invest in combating desertification and to help them identify and characterize the impact indicators to be aggregated at this level.
• For scientists, as evaluation provides keys to perceiving, analysing and understanding a complex reality.

Why is it hard to evaluate the impact of projects?

1. The diversity of projects and actions to combat desertification (objectives, scales, contexts, actors…) imposes the need for appropriate methods to evaluate impact. The impacts directly or indirectly attributable to the project also vary (technical, social, economic, ecological, etc.). This diversity means that no single, remote, a posteriori method of evaluation can be established, nor can any one list of indicators be applicable in all cases.
2. The natural and social systems evaluated are complex and poorly understood, as are the relation(s) of causality between resource use and management practices (vegetation, soils, water, etc.). The resources concerned may be misidentified and poorly visible (e.g. soil organic matter).
3. The stakeholders are complex (types, individual and collective thinking, behaviours, roles, attitudes, interests, points of view…). The set of actors is heterogeneous and fluctuates over time (from local to international). Centres of influence shift and multiply, contradict one another. New actors emerge. This complexity is difficult to grasp through indicators, which by nature, should simplify it.
4. The context of projects to combat desertification is constantly evolving. Natural biological systems, subject to environmental changes and anthropic pressure, evolve. The social changes that accompany projects take place at unpredictable rates. Another aspect to take into account is the evolution of institutional means and of those engaged in combating desertification, which also have local impacts.
5. The spatiotemporal scales of evaluation are hard to define. Desertification is the result of complex, interactive mechanisms and processes, driven by a combination of factors acting at different spatial levels and time scales (which are often long with respect to the dynamics of ecosystems and societies). The unexpected, undesirable, indirect etc. effects induced by project interventions must also be integrated. Such impacts may also occur elsewhere (positive or negative externalities) and at different times.
6. It can be difficult to interpret the results of the evaluation, which may have very different meanings depending on the angle of analysis. Assessment focusing on the evolution of natural resources, for instance, may conclude that a project has failed (or had negative impacts) while evaluation focusing on the economic impact may conclude that it was a success (positive impacts). Moreover, some impacts are unexpected.
7. There are difficulties associated with the availability and quality of the data required for the evaluation. In particular, the lack of documented benchmarks is a common problem in evaluating projects to combat desertification. It underlines the importance of conducting an ex-ante evaluation of the project (initial situation) and of defining a baseline (“without project” situation) to assess the changes and measure the impacts.

Elements of methodology

• Define an initial situation to be able to compare “before project” and “after project” situations. This implies repeating comparable surveys over time, before and after the project.
• Define a baseline “without project” situation to compare the “with project” situation at a given time and measure its impacts. This requires a survey of two analogous samples of individuals or communities in the same area (control and beneficiaries). Failing this, a model or a set of assumptions based on “experts’ statements” is drawn up describing the likely evolution of the initial situation without the project.
• The evaluation must be contextualized. The context has an influence on the impacts of action to combat desertification, particularly their sustainability. Contextualizing consists of characterizing the area where the assessed project takes place. An evaluation will be all the more relevant and informative to stakeholders if it is placed in context, including the legal, political and institutional context. Contextualized evaluation does not allow for aggregation, extrapolation and comparison over time and space as the context is constantly changing. Thus, each evaluation must correspond to a characterization step of the context of the project.
• The approaches used must be holistic, multidisciplinary, intersectorial and systemic. The causes of desertification are complex and must be sought beyond the technical field: poverty, land tenure insecurity, incoherent sectorial policies, etc. The fight against desertification must be approached from all aspects.
Evaluation of projects to combat desertification thus requires appropriate methodologies and approaches involving different components (human, economic, financial, ecological, as well as the practices, techniques and methods implemented…).
• The aspects of land tenure and access to resources must be integrated into the evaluations. Property rights and rights of access to resources affect the way the latter are used. One can understand the reasons for the inefficiency of certain operations to combat desertification when we examine the effects of property rights on the behaviours of the actors (e.g. a process of extensive clearing may be for the purposes of securing land).
• Evaluations must take account of the different local actors involved in combating desertification when designing the system of evaluation (including the choice of indicators), collecting data and analysing results. In fact:
– A project is a collective effort. Each institutional actor has a share of responsibility in implementing and making their own assessment of the phenomena, dynamics and transformations brought about by the project. At the intervention planning stage, shared points of reference must be defined by consensus in relation to the initial situation and to the desired outcome situation.
– Impact analysis is based on reading and interpreting a number of indicators that are parameters agreed upon by the different actors. They must provide a concrete description of a state that has been reached (how, how much, when, who, where).
– The assessment of impacts attributed to the intervention must be solidly argued, based on the analysis of the various beneficiaries of the action and the stakeholders of the intervention. It must also be founded on the independent observations of assessors, based on reading the changes in impact indicators between the initial situation and the situation at the time of the evaluation.
– The assessment process helps to clarify the actors’ roles and interests, linking evaluation with the process of decentralization. More generally, decentralized evaluation is a powerful self-training tool, building the capacity of local actors to manage local resources in their territory. In addition, by providing tangible evidence of impact, evaluation provides an opportunity for actors engaged in combating desertification to form their own opinions.
• The cost of evaluation must be taken into account. The choice of evaluation method and indicators will depend, among other things, on availability of data, budget and the time allocated. Simple, inexpensive tools and methods need to be developed with local management actors. Data gathering is more effective when the indicators are set up by those who actually provide the data. Therefore sufficient resources must be available and the appropriate means and skills implemented through local capacity building (farmers, shepherds, etc.).

The impact indicators selected by the CSFD

There is no universal list of local impact indicators (or evaluation method), however the list must integrate different components: human, economic, financial, practices, techniques and methods of implementation. This is why the CSFD has defined four main types of indicators that combine all the possible impacts. It should be emphasized that the list of local impact indicators proposed by the CSFD is an indicative list. The assumption made is that a few, simple indicators can be combined to help understand certain changes in human and natural environments under the impact of projects to combat desertification. These indicators can be very useful in assessing the natural capital, human capital and social capital of a given area and the evolution of these constitutive parameters of sustainable development.

Four types of local impact indicators
• Biophysical indicators are used to qualify and quantify the status of natural capital (water, soil, vegetation).
• Quantitative indicators of production are used to measure the result of the actions of a project in terms of agricultural production, livestock breeding and forestry products.
• Economic and financial indicators are used to measure investments, funding sources, rates of return etc.
• Institutional and social indicators are used to provide qualitative information about the nature and existence of local agreements and contracts between development actors: farmers and livestock breeders, breeders and technical services. This includes quantitative and qualitative information about how civil society is organized and decentralization; issues of poverty and wellbeing are also addressed (schooling, health…).