Background and context
The United Nations Development Programme (UNDP) Afghanistan Country Office’s Peace Pillar with financial contributions from the Royal Danish Embassy in Afghanistan commenced the Afghanistan Anti-Corruption, Transparency, Integrity and Openness (ACTION) Project on 07 August 2019 with the current end date on 31 December 2024 to allow more time to complete the planned activities.
ACTION project was a continued joint effort to support both institutional (supply) and civilian engagement (demand) sides to increase public trust in and transparency of Afghan security and justice institutions. On the government institutions’ side, under the overall guidance from the Afghanistan Attorney General’s Office (AGO), the project focused its support to the flagship initiative of the government to combat corruption, namely the Anti-Corruption Justice Center (ACJC) and newly established Anti-Corruption Commission (ACC) by enhancing their logistical and technical capacity to address high-level corruption cases. On the demand side, the project is working with civil society organizations and media to improve public awareness on anti-corruption and how to hold service provider accountable for better service delivery. The project started with three outputs and in July 2021, two additional outputs were added as mentioned below:
Output 1: The Anti-Corruption Justice Center Improved Technical and Operational Capacities to Effectively Adjudicate Corruption Cases
Output 2: Non-State Actors and Community Groups, including women and youth, can better promote Transparency, Accountability and Integrity in the Security and Justice Sectors.
Output 3: COVID-19 response’s Transparency and Accountability is strengthened through citizen monitoring
New Output 4: The AGO has improved internal integrity, more efficient processes, and is better able to serve the public
New Output 5: The ACC is strengthened to provide more effective oversight of justice and security institutions for increased prevention of corruption and enhanced service provision.
The Royal Danish Embassy supported the initial three outputs, and the rest were funded by UNDP’s TRAC and other resources.
Under Output 1, ACTION project implemented the capacity building activities for ACJC by delivering 28 training modules identified through a comprehensive training needs assessment. During 2020 and 2021, 110 ACJC judges, prosecutors and technical staff including 15 females were trained. For enabling better working environment for ACJC, the ACTION project also provided resource support to the ACJC. The items of logistics and equipment support to ACJC have been delivered which included new office space, security cameras, kindergarten with equipment, printers, clinic items, sim cards for improved their efficiency and work productivity.
Under output 2, supporting the demand side (civic engagement) ACTION project successfully implemented 27 small grants projects (22 projects Denmark fund and 5 projects UNDP TRAC fund) to build trust between the Afghan community and the state through mobilizing local community on how to fight corruption, conducting community based monitoring of health facilities, social audit of service delivery, reporting corruption cases by media and establishing community based volunteer network to represent community in identifying and resolving corruption cases through active coordination with local and provincial authorities. Through these grants, communities have been mobilized and awareness have been generated on anti-corruption activities by providing information on access to information law. Under investitive journalism 22 local journalists were trained and they produced 55 investigative reports on different corruption cases.
Under Output 3, The project is working in 34 Provinces across 1500 (Primary Health Center) PHCs to monitor the health services through community members related to the support provided by government and other donors to address COVID-19. ACTION M&E COVID-19 app has been developed to create a dashboard by feeding all these data live collected through the partners.
No activity under output 4&5 have been initiated yet after the approval of the revised project document in August 2021.
Due to the recent political changes in the country (the government takeover by the Taliban) and because there are no government partners available anymore to support like the ACJC, the Attorney General’s Office (AGO), as per the instructions by United Nations, the ACTION project will not continue work on activities under output 1, 4 and 5 which are relevant to the government institutions. Out of the above three outputs, the Royal Danish Embassy in Afghanistan supported output 1.
ACTION project will continue the activities under output 2 and output 3 on the demand side engaging with civil society organizations, media, academia, and the private sector.
The Anti-Corruption, Transparency, Integrity and Openness (ACTION) project has been implemented since July 2019. It was originally envisaged to end on 31 December 2020 but was extended through a project revision until December 2021. The project was subsequently revised and extended until 31 December 2024.
Basic Project information can also be included in table format as follows:
PROJECT/OUTCOME INFORMATION |
||
Project/outcome title |
The Anti-Corruption, Transparency, Integrity and Openness (ACTION) |
|
Atlas ID |
00120040 |
|
Corporate outcome and output |
Outcome 2: Accelerate structural transformation for sustainable development |
|
Country |
Afghanistan |
|
Region |
Asia |
|
Date project document signed |
01 July 2019 |
|
Project dates |
Start |
Planned end |
01 July 2019 |
31 December 2024 |
|
Project budget |
US$ 7,025,000 |
|
Project expenditure at the time of evaluation |
US$ 6,075,000 |
|
Funding source |
Denmark |
|
Implementing party |
UNDP/ACTION team (Direct Implementation Modality) |
The Purpose and Objectives of the Midterm Evaluation
The Midterm Evaluation (MTE) aims to inform UNDP Afghanistan and its partners of lessons learned, results achieved and areas for improvements. The MTR will draw out progress toward project deliverables, identify gaps in programming, and any course correction required for the future programming. Furthermore, the findings of MTR will inform the future designing of UDNP’s work. The MTR will be able to produce valuable lessons and experiences, providing useful findings to the other relevant projects and various initiatives organized by UNDP Afghanistan as well as Country Offices (COs) globally. Also, MTR will be able to produce a report on the achievements of the ACTION project plus success stories on the project, which will be published on UNDP website.
Responding to the Theory of Change (ToC) as described in the project document, the agreed results, and resources framework (RRF) and the approved workplans, the MTR should look at the relevance of the project, quality of the project design, effectiveness, and efficiency of the implementation to date, sustainability of the overall project results, impact of intervention made to date, and forward-looking directions for future. To meet these ends, MTR will serve to:
- assess project performance and progress against the expected outputs, targets including indicators presented in the RRF and contribution to expected outcome.
- review and document the success and draw out lessons for deepening impact
- assess the effectiveness of the project’s engagement with UNDP ACTION project implementing partners including civil society organizations (CSOs).
- identify challenges and the effectiveness of the strategic approaches that project adopted for addressing those challenges.
- ascertain the relevance, effectiveness, efficiency, and sustainability of the project interventions.
- outline recommendations, including potential realignments in scope and approach in line with the project’s desired outcome
- provide forward looking recommendations to inform the future designing of UNDP’s work on Anti-corruption area.
- report on the achievements of the ACTION project plus success stories on the project, which will be published on UNDP website.
- highlight project’s overall accountability within the corporate framework, e.g., contribution to higher level results, such as Strategic Plan, CPD etc.
The Scope of the MTR
The MTR is expected to assess the ACTION project progress against the Project Document, targets stipulated in the RRF and the achieved results from 1 July 2019 to 31 December 2022 and propose recommendation which will inform and help improving the coming implementation of the project and designing any future projects. The MTR will be based on a desk review of project related documents and in-depth virtual interviews and surveys as outlined in the methodology section. The MTR will also intend to document achievements, good practices, success stories, lessons learned or transferable examples. Based on the achievements to the date, the MTR will provide forward looking programmatic recommendations for the project’s next months, using the OECD/DAC evaluation criteria on (a) relevance; (b) effectiveness; (c) efficiency; and (d) sustainability
and cross-cutting issues including gender equality.
The MTR’s geographical coverage includes the project’s targeted provinces of Afghanistan. Will be provided to the consultants during the implementation stage.
- Evaluation criteria and key guiding questions
In responding to the MTR purpose and objectives, the MTR criteria and guiding questions can be outline below:
Table 2 – Criteria and Guiding Questions
Criteria |
Guiding Questions |
Relevance |
Relevance of the project: review the progress against project outputs and contribution to outcome level results as defined in the project’s ToC whether assumptions and risks remain valid. Identify any other intended or unintended, positive, or negative, results using following guiding questions.
|
Effectiveness
|
Effectiveness of implementation approaches: review project’s technical as well as operational approaches, the regionality and deliverables, quality of results and their impact, alignment with national priorities and responding to the needs of the CSOs; covering the results achieved, the partnerships established, as well as issues of capacity using following guiding questions.
|
Efficiency
|
Efficiency of the project management structure and the added value of the project’s regional approach: review planning, management, monitoring and quality assurance mechanisms for the delivery of the project interventions and the added value of the regionality of the project set up in the context of fiscal reform at national and subnational level using following questions.
|
Sustainability
|
Sustainability of the project results and risks along with opportunities related to future interventions: review and assess if the current project setup has plans for future resource mobilization, synergy, long term partnership and / or considering institutionalization of the project impact for continued support after the project end using following questions.
|
Diversity and inclusion |
Sustainability and effectiveness of diversity and inclusion approach: review the project’s approaches and strategies in integrating gender and social inclusion (GSI) in the countries, using following questions.
|
Principled |
|
Management and Monitoring |
|
- Methodology
The TOR may suggest an overall approach and method for conducting the evaluation, as well as data sources and tools that will likely yield the most reliable and valid answers to the evaluation questions within the limits of resources. However, final decisions about the specific design and methods for the evaluation should emerge from consultations among the UNDP ACTION project team the evaluators about what is appropriate and feasible to meet the evaluation purpose and objectives and answer the evaluation questions, given limitations of budget, time and data.
Methodological approaches may include some or all of the following:
- Evaluation should employ a combination of both qualitative and quantitative evaluation methods and instruments.
- Document review of all relevant documentation. This would include a review of inter alia
- Project document (contribution agreement).
- Theory of change and results framework.
- Programme and project quality assurance reports.
- Annual workplans.
- Activity designs.
- Consolidated quarterly and annual reports.
- Results-oriented monitoring report.
- Highlights of project board meetings.
- Technical/financial monitoring reports.
- Semi-structured interviews with key representatives of key civil society organizations, and implementing partners:
- Development of evaluation questions around relevance, effectiveness, efficiency, and sustainability and designed for different UNDP ACTION project implementing partners to be interviewed.
- Key informant and focus group discussions with men and women, beneficiaries of UNDP ACTION project implementing partners.
- All interviews should be undertaken in full confidence and anonymity. The final evaluation report should not assign specific comments to individuals.
- Surveys and questionnaires including participants in development programmes, surveys and questionnaires involving UNDP ACTION project implementing partners at strategic and programmatic levels.
- Field visits and on-site validation of key tangible outputs and interventions.
- The evaluator is expected to follow a participatory and consultative approach that ensures close engagement with the evaluation managers, implementing partners and direct beneficiaries.
- Other methods such as outcome mapping, observational visits, group discussions, etc.
- Data review and analysis of monitoring and other data sources and methods.
- Ensure maximum validity, reliability of data (quality) and promote use; the evaluation team will ensure triangulation of the various data sources.
As part of the requirement, evaluation must include an assessment of the extent to which the design, implementation, and results of the project have incorporated gender equality perspective and rights-based approach. The evaluators will make sure Integrating Human Rights and Gender Equality in Evaluation during the inception phase. In addition, the methodology used in the evaluation, including data collection and analysis methods should be human rights and gender-sensitive to the greatest extent possible, with evaluation data and findings disaggregated by sex, ethnicity, age, etc. – with a focus on people with disabilities. Detailed analysis on disaggregated data will be undertaken as part of final evaluation from which findings are consolidated to make recommendations and identify lessons learned for enhanced gender responsive and rights-based approach of the project. These evaluation approach and methodology should consider different types of groups in the project intervention – women, youth, vulnerable groups etc.
The final methodological approach including interview schedule, field visits and data to be used in the evaluation should be clearly outlined in the inception report and be fully discussed and agreed between UNDP and the evaluators.
- Evaluation products (deliverables)
The terms of reference should clearly outline the outputs UNDP expects from the evaluation team as well as a detailed timeline and schedule for completion evaluation products. Where relevant, the TOR should also detail the length of specific products (pages). These products could include:
- Evaluation inception report (10-15 pages). The inception report should be carried out following and based on preliminary discussions with UNDP after the desk review and should be produced before the evaluation starts (before any formal evaluation interviews, survey distribution or field visits) and prior to the country visit in the case of international evaluators.
- Evaluation debriefings. Immediately following an evaluation, UNDP may ask for a preliminary debriefing and findings.
- Draft evaluation report (within an agreed length). The ACTION project team should review the draft evaluation report and provide an amalgamated set of comments to the evaluator within an agreed period, addressing the content required (as agreed in the TOR and inception report) and quality criteria as outlined in these guidelines.
- Evaluation report audit trail. Comments and changes by the evaluator in response to the draft report should be retained by the evaluator to show how they have addressed comments.
- Final evaluation report, reflecting the achievements and success stories of the project.
- Presentations to NDP ACTION project team
Expected Deliverables and Descriptions
# |
Deliverables |
Description |
Due date |
||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 |
ACTION project MTR Inception report |
The workplan should provide clear timeline of how each MTR steps will be undertaken. The consultant is required to provide clear interview and/or focus group discussion scheduled online as this will required coordination support from the ACTION project team at Kabul. The ACTION project MTR methodology should provide a specific assessment framework, covering both quantitative and qualitative dimensions, with a detailed list of required UNDP ACTION project implementing partners who need to be interviewed in the MTR process. A simple UNDP ACTION project implementing partners analysis for conducting interviews and evaluations can be conducted. The draft methodology can be adjusted later once the MTR consultants have completed the desk review of the project related documents. The final MTR approach and methodology can be presented as a part of the Inception Report. |
8 days after the contract signed |
||||||||||||||
2 |
MTR evaluation briefing |
After completion of data collection or before sharing the draft report, the evaluator should present preliminary debriefing and findings to UNDP and MTR reference group. |
10 days after the contract signed |
||||||||||||||
3 |
Draft MTR report |
|
30 days after the contract signed |
||||||||||||||
4 |
Final MTR report with achievements and success stories of ACTION project |
|
50 days after the contract signed |
||||||||||||||
5 |
Audit Trail Form |
The comments and changes by the consultant/s in response to the draft report should be retained by the evaluator in form of audit trial to show they have addressed comments. This document can be submitted as an Annex to the final evaluation report. |
60 days after the contract signed |
Payment Modality:
Payment to the individual contractor will be made based on the actual number of days worked, deliverables accepted and upon certification of satisfactory completion by the manager.
payment Instalments:
Deliverables/ Outputs |
Estimated Working Days |
% Of total contract amount |
|
1st Instalment |
Upon satisfactory completion of the MTR inception report (including final methodology, data collection tools and questions, proposed data collection schedules, evaluation matrix, evaluation briefing etc.) |
8 days |
30% |
2nd Instalment |
Upon satisfactory completion of desk review, interviews and analysis and submission of MTR draft report including debrief on draft findings and recommendation to the management |
30 days |
40% |
3rd Instalment |
Upon satisfactory submission of MTR Final Draft and completion of final presentation. |
60 days |
30% |
Total |
60 days (43 working days) |
100% |
* The total duration of the task should not exceed 25 working days.
* A buffer of 5 days is given for both consultants for unforeseen circumstances.
Consultant must send a financial proposal based on per-diem The total amount quoted shall be all-inclusive and include all costs components required to perform the deliverables identified in the TOR, including professional fee, living allowance and any other applicable cost to be incurred by the consultant in completing the assignment. The contract price will be fixed output-based price regardless of extension of the herein specified duration. Payments will be done upon completion of the deliverables/outputs and as per the above percentages
It is important to note that multiple iterations of the report may be required for the satisfactory completion of the report.
In general, UNDP shall not accept travel costs exceeding those of an economy class ticket. Should the consultant wish to travel on a higher class he/she should do so using their own resources.
In the event of unforeseeable travel not anticipated in this TOR, payment of travel costs including tickets, lodging and terminal expenses should be agreed upon between the respective business unit and the Individual Consultant prior to travel and will be reimbursed.
Travel costs shall be reimbursed at actual but not exceeding the quotation from UNDP approved travel agent.
Implementation arrangements
The ACTION project team will be in touch with the MTR consultant/s and help with the day-to-day coordination for MTR process with ACTION project partners. The details of the implementation arrangement are described in Table 3.
The MTR Consultant/s will be briefed by UNDP ACTION project team upon arrival on the objectives, purpose, and output of the evaluation. An oral debriefing by the MTR Consultant on the proposed work plan and methodology will be done and approved prior to the commencement of the process.
The ACTION MTR will remain fully independent and reports to ACTION project Manager at UNDP Afghanistan. The MTR Consultant/s maintains all the communication through assigned ACTION project team during the implementation of the evaluation. The Evaluation Manager should clear each step of the evaluation. Evaluation report must meet the requirements from the Independent Evaluation Office’s guidelines which will be provided as part of the inception meeting.
Contractors will arrange online final presentation with UNDP ACTION team and noted comments from participants which will be incorporated in the final report.
It is understood that it may take multiple rounds of feedback before Evaluation Report is finalized and approved.
The final report will be signed off by Evaluation Manager.
Time frame for the evaluation process
ACTIVITY |
ESTIMATED # OF DAYS |
DATE OF COMPLETION |
PLACE |
RESPONSIBLE PARTY |
Phase One: Desk review and inception report |
||||
Meeting briefing with UNDP (ACTION Project Manager) |
1 day |
At the time of contract signing 25 September 2022 |
UNDP or remote |
ACTION project team |
Sharing of the relevant documentation with the evaluation team |
– |
At the time of contract signing 1 October 2022 |
Via email |
ACTION project team |
Desk review, Evaluation design, methodology and updated workplan including the list of CSOs representatives to be interviewed |
9 days |
Within 10 days of contract signing 1 to 10 October 2022 |
Home- based |
Evaluation team |
Submission of the inception report (15 pages maximum) |
– |
Within 10 days of contract signing 1 to 10 October 2022 |
UNDP |
Evaluation team |
Comments and approval of inception report |
2 days |
Within 12 days of contract signing 1 to 12 October 2022 |
UNDP |
Evaluation team and ACTION team |
Phase Two: Data-collection mission |
||||
MTR evaluation briefing |
1day |
Within 13 days of contract signing 1 to 13 October 2022 |
In country
With field visits |
ACTION project team to organize with local project partners and, NGOs, etc. |
Debriefing to UNDP ACTION project team |
8 days |
21 October 20228 |
In country |
Evaluation team |
Phase Three: Evaluation report writing |
||||
Preparation of draft evaluation report (50 pages maximum excluding annexes), executive summary (5 pages) |
21 days |
Within three weeks of the completion debriefing to UNDP ACTION project team 21 October to 12 November 2022 |
Home- based |
Evaluation team |
Draft report submission |
1 day |
13 November 2022 |
|
Evaluation team |
Consolidated UNDP comments to the draft report |
5 days |
Within five days of submission of the draft evaluation report 18 November 2022 |
UNDP |
Evaluation team |
Debriefing with UNDP |
7 days |
Within 1 week of receipt of comments 24 November 2022 |
Remotely UNDP |
Evaluation team and ACTION project team |
Finalization of the evaluation report incorporating additions and comments provided by project staff |
3 days |
Within 3 days of final debriefing 27 November 2022 |
Home- based |
Evaluation team |
Submission of the final evaluation report to UNDP ACTION project team (50 pages maximum excluding executive summary and annexes) |
3 days |
Within one week of final debriefing 30 November 2022
|
Home- based |
Evaluation team |
Estimated total days for the evaluation |
60 |
|
|
|
Application submission process and criteria for selection
Proposal Evaluation Method and Criteria:
The award of the contract shall be made to the individual consultant/s whose offer has been evaluated and determined as:
- Responsive/compliant/acceptable; and
- Having received the highest score out of a pre-determined set of weighted technical and financial criteria specific to the solicitation.
Technical Criteria weight 70%
Financial Criteria weight 30%
Only candidates obtaining a minimum of 49 points (70% of the total technical points) would be considered for the Financial Evaluation.
Technical Criteria 70 points
Technical Proposal (30 marks)
- Technical Approach & Methodology (20 marks) – Explain the understanding of the objectives of the assignment, approach to the services, methodology for carrying out the activities and obtaining the expected output, and the degree of detail of such output. The Applicant should also explain the methodologies proposed to adopt and highlight the compatibility of those methodologies with the proposed approach.
- Work Plan (10 marks) – The Applicant should propose the main activities of the assignment, their content and duration, phasing and interrelations, milestones (including interim approvals by the Client), and delivery dates. The proposed work plan should be consistent with the technical approach and methodology, showing understanding of the TOR and ability to translate them into a feasible working plan.
Qualification and Experience (40 marks) [evaluation of CV]:
- General Qualification (15 marks)
- Experience relevant to the assignment (25 marks)
Documents to be included when submitting the proposals:
Interested individual consultant/s must submit the following documents/information to demonstrate their qualifications in one single PDF document:
- Duly accomplished confirmation of Interest and Submission of Financial Proposal
- Personal CV or P11, indicating all experience from similar projects, as well as the contact details (email and telephone number) of the candidate and at least three (3) professional references.
Technical Proposal:
- Brief description of why the individual considers him/herself as the most suitable for the assignment
A methodology, on how they will approach and complete the assignment and work plan as indicated above.
The consultant/s shall submit a price proposal as below:
- Daily Fee: The consultant shall propose a daily fee which should be inclusive of his/her professional fee, local communication cost and insurance (inclusive of medical evacuation). The number of working days for which the daily fee shall be payable under the contract is 21 working days.
- The consultant is NOT allowed to stay in a place of his choice other than the UNDSS approved places. The payment of accommodation shall be made directly by the Project.
- Travel and Visa: The consultant shall propose an estimated lump sum for home-Kabul-home travel and Afghanistan visa expenses. The ACTION project will cover the cost of internal travel within Afghanistan.
The total professional fee shall be converted into a lump sum contract and payments under the contract shall be made on submission and acceptance of deliverables under the contract in accordance with the abovementioned schedule of payment. The total professional fee shall be converted into a lump sum contract and payments under the contract shall be made on submission and acceptance of deliverables under the contract in accordance with the abovementioned schedule of payment.
TOR annexes
Annexes can be used to provide additional detail about evaluation background and requirements to facilitate the work of evaluators. Some examples include:
- Intervention results framework and theory of change. Provides more detailed information on the intervention being evaluated.
- Key partners. A list of key UNDP ACTION project implementing partners and other individuals who should be consulted, together with an indication of their affiliation and relevance for the evaluation and their contact information. This annex can also suggest sites to be visited.
- Documents to be consulted. A list of important documents and web pages that the evaluators should read at the outset of the evaluation and before finalizing the evaluation design and the inception report. This should be limited to the critical information that the evaluation team needs. Data sources and documents may include:
- Relevant national strategy documents,
- Strategic and other planning documents (e.g., programme and project documents).
- Monitoring plans and indicators.
- Partnership arrangements (e.g., agreements of cooperation with Governments or partners).
- Previous evaluations and assessments.
- UNDP evaluation policy, UNDP norms and standards and other policy documents.
- Evaluation matrix (suggested as a deliverable to be included in the inception report). The evaluation matrix is a tool that evaluators create as map and reference in planning and conducting an evaluation. It also serves as a useful tool for summarizing and visually presenting the evaluation design and methodology for discussions with UNDP ACTION project implementing partners. It details evaluation questions that the evaluation will answer, data sources, data collection, analysis tools or methods appropriate for each data source, and the standard or measure by which each question will be evaluated.
Sample evaluation matrix
Relevant evaluation criteria |
Key questions |
Specific sub questions |
Data sources |
Data-collection methods/tools |
Indicators/ success standard |
Methods for data analysis |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
- Schedule of tasks, milestones and deliverables. Based on the time frame specified in the TOR, the evaluators present the detailed schedule.
- Required format for the evaluation report. The final report must include, but not necessarily be limited to, the elements outlined in the quality criteria for evaluation reports.
- Code of conduct. UNDP ACTION project team should request each member of the evaluation team to read carefully, understand and sign the ‘Code of Conduct for Evaluators in the United Nations system’, which may be made available as an attachment to the evaluation report.