Christian Aid: Consultancy for a Baseline Measurement for CA Ireland

The Programme Grant (PG) II is a governance, human rights and peace building programme spanning six country/regional programmes, and funded by Irish Aid for 2017-2021.1 These countries, many of whom are experiencing or have recently experienced violent conflict, are: Colombia, Guatemala, El Salvador, Sierra Leone, Angola, Zimbabwe and Israel and the occupied Palestinian territory. PG II has been designed based on Christian Aid’s understanding of social change as a long-term, unpredictable process, where development interventions can make a contribution to certain expected outcomes, but will do so as part of a number of concerted actions by civil society actors, the private sector, governments, and citizens as individuals or as part of movements. Christian Aid Ireland’s programme focus is on poor and marginalised people, and recognises that each person will experience poverty and marginalisation differently depending on their gender, age, disability, ethnic group and other intersecting factors.

The programme recognises that development is complex and change is unpredictable and often non- linear. The PG II uses a Theory of Change approach of systematic critical reflection and a learning-by- doing approach that is deliberate, structured and subject to regular and focused monitoring. The programme uses adaptive management - best understood as a broad approach, rather than as a specific method or set of tools. The key distinguishing features of an adaptive approach are that it is applied to work on complex problems whose nature and solution cannot be predicted in advance, and that it involves using feedback, evidence and learning to make ongoing course-corrections in interventions. The programme theory of change is based on the assumption that the best outcomes will be achieved if the programme is flexible enough to adapt to changing and challenging contexts. Strategies chosen at the start of the programme must be able to change if or when opportunities for better effectiveness or greater change emerge.

In order to support this approach, the programme M&E system supports an adaptive programme design. Data collection and analysis has a dual purpose of a) informing programme management and learning, including an annual Strategy Testing process where Theories of Change will be reviewed, 2 and b) assessing progress against the outcomes for accountability and reporting. While ongoing data

1 This is a continuation of the programme from 2012-2016.
2 For this programme, Christian Aid Ireland has developed a TOC based adaptive programming approach, using
a method called Strategy Testing (adapted from content/uploads/2015/10/Strategy-Testing-An-Innovative-Approach-to-Monitoring-Highly-Flexible-Aid- Programs.pdf). 
collection is based on M&E at the level of partner organisations, complemented by Outcome Harvesting,3, we will also establish a baseline for all programme outcomes, using specific indicators of change, against which a periodic comparison at midpoint and at endpoint can be made to assess before-after change caused by the programme.

This invitation to tender focuses on the development of these baselines, which will allow a before- after assessment of change in relation to the outcomes that the programme is designed to achieve. Christian Aid Ireland invites tenders from interested external consultants who will lead the data collection processes at country level and provide coherency across the programme.

Desired Approach
The programme baseline will serve to inform adaptive decision-making during programme management, but it will also allow us to report back to the donor, to Christian Aid, to partners and to primary stakeholders on what the programme has achieved. Its development and the monitoring against it form part of the overall compliance commitments of the programme.

The baseline will be designed to provide information on a country by country basis, and to assess progress against the country outcomes, though using a common set of programme indicators. Each country baseline will have two main parts:

A)   Overview of the context: This should provide a short assessment of the political, social and cultural and economic context within which the programme is working, with a particular focus on the situation of poor and marginalised women and men and other excluded groups for each country programme outcome.  This overview summary should provide enough background to ensure that results achieved by the project can be understood within the context – for example, this may be referred to later at mid-line and end-line to assess the extent to which the programme has been affected by the context, either politically, economically, culturally or socially.
B)   Baseline data: The baseline data should then allow an assessment of progress towards
programme outcomes as defined in the Programme Results Framework, specific to each country context but reflecting closely the overall theory of change of the programme. For each country, the baseline will provide information against each programme indicator, either qualitatively or, in some cases, quantitatively. The emphasis will be on measuring outcome level change, rather than measuring the delivery of the programme strategies.

Programme progress will be monitored against the baseline at midpoint and towards the end of the programme. In order to ensure the midline and endline will be comparable, it is assumed that the same data gathering methods will be applied at the two following points.4  For this reason, an introduction of country teams and partner staff to the methods used for the baseline data gathering will be an important part of the approach of this assignment.

3 This programme is using Outcome Harvesting as its main ongoing data gathering methodology for measuring change (adapted from saferworld-learning-paper-2016.pdf).
4 In some cases, partners and Christian Aid country staff may choose to repeat the data gathering at an annual
interval, if resources and logistics allow. 
A Theory of Change Based Approach
As mentioned above, the programme has been designed on the basis of Christian Aid’s analysis about how change happens, set out in the Programme Theory of Change (See Figure 1). Essentially, we believe that in order to make society more just and equitable in the long term we must alter the balance of power in our societies and transform inequitable access to rights and resources, and the deeply embedded norms that perpetuate inequality and poverty.5 Each country has then developed a country level theory of change based on the national context, which sets out what change is expected, what is needed for the change to come about, and suggests initial strategies partners will adopt to achieve the change. These country level theories of change are underpinned by partner level theories of change, which provide additional analysis and detail about how the programme will achieve the expected outcomes.

Figure 1: The programme level Theory of Change

In order to measure change at a programmatic level, a set of 11 Programme Indicators were developed to capture the change we want to see under outcomes 1-3.6 These indicators will form the structure

5 See also Christian Aid Ireland’s  Programme Strategy 2016-21 for more detail.
6 The Programme Outcome 4, on Public Engagement, will be subject of a separate baseline development
for data collection in the baselines, and should enable a consistent monitoring of change across the programme. See Box 1 for a list of the 11 Programme Indicators.

Box 1: Key Indicators for Programme Outcomes 1-3 of the PG II
Key Programme Indicators at micro, meso and macro levels7
1.   Micro: Evidence that primary stakeholders (numbers of, disagg) have retained or gained access to basic services, land, housing, livelihood opportunities and social protection as a result of programme initiatives
2.   Micro: Extent to which primary stakeholders (disagg) are satisfied with/confident in their ability to claim rights and hold power holders to account
3.   Meso+macro: Evidence that decision-makers and power holders at local and national level,
influenced  by  primary  stakeholders  and  their  representatives,  have  become  more accountable and responsive
4.   Meso+macro: Evidence that policy reform, legal processes and fiscal decisions, and their implementation, have become more progressive, inclusive and gender sensitive as a result of programme initiatives
5.   Micro: Extent to which women and men (incl. HRDs) have been protected from violence in the public and private spheres (incl. GBV)
6.   Meso: Level of reported incidents of violence in programme areas
7.   Meso: Evidence that duty bearers have responded to citizens/civil society calls for greater safety, protection and peace
8.   Meso: Evidence of (and numbers of) HROs, church and community leaders challenging violence and impunity for human rights violations
9.   Meso+macro: Evidence that peacebuilding/conflict transformation processes influenced by partner engagement are more inclusive, gender sensitive and/or sustainable
10. Meso: Evidence of women actively involved in decision-making structures as a result of programme initiatives
11. Micro+meso+macro: Evidence that GBV is being challenged and addressed, and that social norms around gender equality have shifted as a result of programme initiatives

Bringing in Community Voices
A fundamental part of the baseline development will be ensuring the accountability of the programme to its primary stakeholders, or the men, women, young people, indigenous, poor, disabled or otherwise marginalised, who stand to benefit in the first instance from the programme. Not only will the programme draw on these views in an inclusive way, in order to monitor progress, but it will need to ensure that there is feedback to the individuals, households and community groups participating in data collection on programme delivery, progress and decision-making. A key part of the baseline therefore will be the use of specific methods in each country to allow the voices of communities and individuals to be a part of the baseline and make a contribution to programme accountability and learning. This approach to data gathering will affect indicators at micro level in the first instance, but communities and individuals might also contribute to the baseline at meso and macro levels. See Box
2 for some possible methodologies to use to bring in Community Voices.

Box 2: Some suggested methods to capture Community Voices in PG II
•   Community based surveys designed using the programme indicators

7 Micro level: Change at individual, household and community level, involving no organisational change; Meso level: Change within and by civil society structures (such as faith-based organisations, CSOs, etc.) and local authorities (eg. policy and practice change); Macro level: Change at national level (eg. legislative, policy and
•    Focus  group  discussions  (disaggregated  by  sex,  age,  (disability))  designed  around  the programme indicators (NB. Data analysis when using FGDs will need to be rigorous - Recording ad hoc statements will not be adequate)
•   Using perception scales within disaggregated focus group discussions
•   Video surveys, eg. the  “ Truth T ruck”  in Nepal
•   Participatory video capturing of community narrative / community responses

The methodology adopted for this baseline will be as follows:

Appointment of consultants
Consultants will be able to tender for one country or for more than one country and an interview process will  assist  in  choosing  consultants.  Decisions  on  appointment will  be  made  collectively between the Programme Performance Advisor (PPA) in Christian Aid Ireland, and country programme staff.

Agreement of the final ToR for each consultant and contract signature will be managed by the country teams if the consultant is based in that country, or by Christian Aid Ireland, when the consultant, or team of consultants, will be working in more than one country, or is not based  in one of the programme countries.

Coordination of the baseline work across the seven countries will lie with a coordination team made up of the Programme Performance Advisor, a representative from the country programmes, and one of the consultants. The coordination role played by the consultant will entail additional work days for this consultant, which will be recognised as such in their contract.

Coordination between the consultants, the coordination team, and the country teams throughout the assignment will be crucial to its success.

Development of the Methodology
Development of the methodology will take place in two stages:

1)   A first discussion will take place between the consultant(s), the Programme Performance Advisor and the country team, to scope the methodology options that seem most appropriate for each country.   In particular, this discussion should focus on what method to use for bringing in Community Voices.
2)   A methodogy workshop will then be held in early May 2017, to agree the quality of data required, the methodology used in each country, and the tools. The workshop will most likely take place in Ireland,8 and the working language will be English.

Preparation of country data collection
Once the methodologies have been agreed, consultants will prepare data collection. Given that in a number of cases data collection will include surveys, this may include identification of enumerators and data input personnel, and their training, or alternatively, means to gather data digitally. Staff from partner organisations may be well placed to participate in this process, as well as in the data collection itself. Where alternative media approaches will be applied, these will also need to be prepared, eg. video production teams need to be recruited and trained.

8 There may be an option to participate virtually, if there are visa or travel issues. 
Data collection in country
This phase will involve three types of data collection:

1)   Information about the political, economic, social and cultural situation in a country, for Part
1 of the baseline – this may include secondary data such as reports, publications, press information, etc. but may also include interviews with key informants.
2)   Community  Voices  data,  which  captures  and  reflects  the  perspectives of  individuals  or
community members who stand to benefit in some way from the programme. This will be most relevant for the micro level indicators in the baseline (see Box 1), and may often take a quantitative approach.
3)   Data (or situational snapshots) which describe the starting point for each of the indicators in
the baseline (see Box 1). It is important to note that one of the key means of data collection during  programme implementation will be a method called  Outcome  Harvesting,  which gathers qualitative outcome statements and analyses them to assess the contribution the programme made towards them.9 There will therefore be a bias towards qualitative data in this type of data.

Each consultant will put together a baseline report. This report will be a maximum of 10 pages of narrative, accompanied by a tabular presentation of data against each of the indicators. This table will need to be concise and focused on comparability over time, as this information will be used to populate the Programme Results Framework that Christian Aid Ireland will use to report to Irish Aid over the five-year programme.

The reports will contain the following sections: A)   Executive Summary (1 page)
B)   Introduction/background (1 page)
C)   Analysis of political, economic, social and cultural situation for each of the three programme outcomes (3-4 pages)
D)   Methodology used (1 -2 pages)
a.   Include process comments (eg. qualifying the validity of data, if necessary) E)   Baseline data (tabular format)
a.    Micro level indicators: provide results of surveys / video analysis / FGD analysis
b.   Meso + macro level indicators: situational snapshot to enable the assessment of change
F)   Comments on findings (1-2 pages)
a.   Include recommendations for subsequent data collection (eg. at midline and endline) G)   Conclusions (1 page)

In addition, each consultant will put together a short manual to describe the methodology used to gather the data. This manual should provide enough information that the process of data collection can be repeated at midline and endline of the programme by someone else. The manual should be accessible to Christian Aid and partner staff (who are not experts in data collection), and user-friendly. Christian Aid and partner staff should have received an introduction to the manual.

9 For more information on Outcome Harvesting please see: and

Both Christian Aid Ireland and country programmes have budgeted for the baseline development. The budget will inform the choice of consultant for each country baseline. Each consultant is invited to submit both a minimum daily rate, and an estimation of how many days they envisage this work taking.

Consultant Specifications

Required skills and competencies for this assignment:

•    Relevant university degree and postgraduate /master degree in social science or political science
•    Excellent track record in using quantitative and qualitative methods in order to conduct baselines, evaluations, or other sense-making exercises; a specialisation in some particular evaluation methods, such as Process Tracing, Outcome Mapping, Outcome Harvesting, Most Significant Change, Qualitative Comparative Analysis (QCA), Contribution Analysis, participatory video evaluations, or other, would be desirable.
•   A good understanding of using Theories of Change in programming; knowledge of Adaptive
Management would be desirable.
•    A good understanding of governance and human rights, conflict and violence, and gender and power in development programming.
•    Proven knowledge of the political, economic, social and cultural context of the country or countries for which a consultant is expressing an interest.
•   Excellent analytical, writing and synthesis skills and evidence of high quality reporting.
•    Excellent communication and writing skills in English if applying for the baseline development in Sierra Leone, Zimbabwe or IOPT, Spanish if applying for the baseline development in Colombia, Guatemala or El Salvador, and Portuguese if applying for the baseline development in Angola. In Sierra Leone and Zimbabwe, ability to communicate in the local language(s) would be desirable.
•   Ability to communicate confidently in English, both spoken and in writing.

To apply for this piece of work, please supply the following information to Alix Tiernan at by the 13th April 2017:

•    A proposal of no more than 2 pages explaining (a) the consultant's (or consultants’) proposed approach and methodology to conducting the baseline (b) availability within the indicated time frame, in particular for a workshop in early May (c) the consultant/s’ competency for the job and (d) both a minimum daily rate, and an estimation of how many days they envisage this work taking.
•    An updated CV of no more than 2 pages for the consultant(s) detailing relevant skills and experience.
•    An example of previous relevant work, if possible showing data analysis of surveys, FGDs, or other community based methodology.
•   One referee from previous relevant work.

Preference will be given to consultants, or consultant teams, who will tender for more than one country; preference will also be given to consultants who have fluency, both spoken and written, in English.

Short-listed candidates will be interviewed (via Skype if outside Ireland) between the 18th  and 21st
April 2017.