Independent external evaluation, Safe in Custody Project

1. About the APT

The Association for the Prevention of Torture (APT, www.apt.ch) works for societies without torture and ill-treatment. Established in 1977, we helped establish a global system to prevent torture by increasing transparency in all places where people are deprived of their liberty.

We advocate for torture prevention at the international, regional and national levels with an approach based on cooperation, dialogue and partnerships. Working with actors of change to achieve solutions that have their full ownership, we strengthen efforts to reduce the risks of torture and ill-treatment across the world.

APT’s vision: Societies without torture or ill-treatment.

APT’s mission:

• Influencing national, regional and international legal frameworks, standards, policies and practices to reduce risks of torture

• Supporting partners’ efforts to advance torture prevention, and contributing our respective expertise and resources to maximise impact

• Bringing people together in constructive dialogue to build support for torture prevention and identify practical solutions

2. About the project

Project title and reference:

“Reducing risky practices leading to torture and ill-treatment in police custody in Malaysia, Philippines and Thailand” project, EIDHR/2020/ 419-929. The project is also commonly referred as “Safe in Custody Project”.

1. Geographical scope and objectives

The project focuses on increasing transparency of the police (obj. 1), improving police practices (obj. 2), and shifting public mindsets about torture (obj. 3) in Malaysia, the Philippines, and Thailand.

Overall objective:  To prevent torture and ill-treatment in Malaysia, the Philippines and Thailand by building new synergies against forced confessions and incommunicado detention.

Specific objective 1: to increase transparency in police custody, by enhancing oversight capacity and synergies 

→ Philippines was identified as focus country under this first objective.

Specific objective 2: to improve police practices by implementing safeguards and non-coercive interviewing 

→ Thailand was identified as focus country under this second objective.

Specific objective 3: to change public mindsets to reject of torture by the police, by increasing knowledge on rights and mobilising artists and youth

→ Malaysia was identified as focus country under this third objective.

2. Summary of the project logic

APT-commissioned outstanding study "Does torture prevention work?" (2016) highlights that torture can be effectively reduced by implementing preventive measures during risky situations.[1] The study evidences that the risks of torture are the highest during the first hours of police custody. In the three project countries, incommunicado detention and forced confessions have been identified as practices of increased risks that can be addressed through prevention. In particular, oversight, implementation of safeguards, non-coercive interviewing and mobilisation can contribute to reducing these risks.

The project concept was built on a) the findings of the study, 2) pilot work in Thailand to implement safeguards between 2018 and 2020, and 3) EU call’s requirements (e.g. work within the same region, no work relating to legal reform, incl. no advocacy for new ratifications).

Finally, APT’s experience shows that trust and cooperation between law enforcement authorities, civil society and national human rights institutions are crucial to achieve real and sustainable changes. This approach has been a major lever for action under this project, e.g. to create synergies that are necessary for achieving results.

3. Result-Oriented Monitoring review and report

Upon request from the EU Delegation in Bangkok (who is the contracting authority on behalf of the European Commission under this project), the project was assessed after one year through a Results-Oriented Monitoring (ROM) process. The purpose of any ROM review is to enhance the EU's accountability and management capacities, provide external opinions on project implementation, and offer recommendations to improve future implementation for both implementing partners on one side and EU delegation on the other side.

The assessment was conducted by a freelance evaluator, contracted through the EU delegation in Bangkok. He based its assessment on the following OECD DAC criteria: relevance, efficiency, effectiveness and sustainability. The period assessed was running from March 2021 (when the project started) until July 2022, so almost 1.5 year and approx. 40% of the project lifetime.

Executive summary of the ROM report

“Strengthening human rights protection is a long-term endeavour that facilitates the development of prosperous societies. The intervention addresses risky practices leading to torture and ill-treatment in police custody in Malaysia, the Philippines, and Thailand.

The complexity of the issue to address, the COVID-19 pandemic, and the fluid and overall non-enabling political environments have posed mounting challenges for a standard implementation and have caused delays. Following 19 months of implementation, the intervention remains powerfully relevant to the needs and rights of end beneficiaries and target groups in the three targeted countries. The choice of Implementing Partners (IPs) is appropriate. The applicant [APT] is a leading force in the global prevention of torture, while the co-applicants [Cross Cultural Foundation in Thailand, SUARAM in Malaysia, and Task Force Detainees in the Philippines] display a long-standing track record as Human Rights Defenders (HRDs) and are pivotal players of the universal human rights framework and natural allies of the United Nations (UN) and EU institutions in their shared vision to push the human rights agenda forward.

The intervention has developed synergies and complementarities with other interventions and reflects on / benefits from the EU's added value as the leading promoter and supporter of HR and democracy across the globe.

The intervention logic is ambitious, comprehensive, and feasible but the logframe presents several shortcomings in both its vertical (results chain) and horizontal (indicators) structure. The monitoring system also has areas for improvement.

The intervention resources currently planned and budgeted are suitable for completing all activities and achieving the set objectives. Staff turnover of the IPs delayed the implementation at the country level and had consequences in the timely delivery of a few Outputs.

The quality of the realised Outputs is perceived as sound by the targeted groups except for the protracted regional component. Activities are undertaken, and Outputs are accomplished through a sound human rights-based approach (HRBA) methodology led by each national IP. The feasibility of reaching the envisaged Outcomes to a positive degree is sound despite the ambitious targets set and the highly challenging environment where the IPs are operating.

The sustainability prospects are sound, given the strengthened capacities of several key target groups, including those of the IPs, to better fulfil their mandates as duty-bearers, responsibility-bearers and rights-holders, the reduction of the gap between laws and applied procedures that ensure the rights of people deprived of liberty and the increased awareness and action of the public, and particularly youth, against torture and ill-treatment by police institutions.

The intervention distinctly adheres to an HRBA and is fully inserted in the UN HR framework. The IPs show commitment to gender equality and provisions are foreseen in the intervention documentation and incorporated in the logframe; however, gender disaggregation data is not collected, analysed or reported.

The intervention complies with the Communication and Visibility requirements for EU External Actions, notwithstanding the need for continuous risk assessment due to the sensitivity of HR in the operation spaces. Several Outputs have sound potential to influence the EU image positively.

Main recommendations: i) improve the logframe shortcomings; ii) reinforce the monitoring and reporting system; iii) maximise efforts on the ongoing introduction of the hope-based communications and effective interviewing approaches; iv) ensure staff stability and increase the leading role of co-applicant IPs in their respective countries; v) ensure the application of gender equality provisions; vi) enhance the forward planning anticipating workload with the Contracting Authority; vii) reflect on providing further guidance to relevant EU Delegations on HR sensitive projects.”

Subsequently, the evaluator supported APT in revising the project logframe. This resulted in the revised logframe, being approved by the EU delegation in Bangkok early 2023.

3. Purpose and objectives of the evaluation

The external, independent evaluation of the project will determine whether and to what extent the APT has contributed to the project objectives, focusing on the achievement of results. The main success factors, challenges and lessons will be identified, as well as opportunities for follow-up and strengthening in the future, and possible disengagement where relevant.

The evaluator will collect evidence of what has changed and whether, how and to what extent the project has contributed to these changes (outcome harvesting approach).

The evaluation aims to:

• Assess the results actually achieved versus what was expected;

• Identify the levers of success and enabling factors;

• Learn lessons for future work (including if possible mitigation of risks and management of challenges);

• Assess the sustainability of the work done and results achieved in Malaysia, Philippines and Thailand.

Considering that the project’s relevance and efficiency have largely been validated by the ROM review, this final evaluation will focus on the effectiveness, the impact and the sustainability of the project.

The evaluator’s recommendations in the evaluation report should be as contextualized and practical as possible to facilitate their application in the future.

4. Scope of the evaluation

Period under evaluation

Entire project period, from 01 March 2021 to 31 August 2024.

5. Geographical scope

The project has three target countries: Malaisia, Philippines, Thailand. 

The implementation of the strategies has not been exactly the same in all priority countries and has varied in intensity from one country to another. This approach builds on national contexts and partners’ capacities. Objective 1 (oversight) focused on the Philippines, Objective 2 (police practices) on Thailand, and Objective 3 (mobilization) on Malaysia.

This strategic, differentiated involvement will be reflected in the inception report and eventually in the evaluation report.

6. Content

The evaluation will cover all the project objectives and the evaluation objectives and criteria as stated above, with priority given to the evaluation criteria on effectiveness, impact and sustainability and on success factors.

In a cross-cutting manner, the evaluation will also assess APT’s preventive approach and methods of work, especially when they relate to partnerships and synergies among stakeholders, and to challenging political environments.

5. Evaluation criteria and methodological approach

The evaluation will assess the project against the OECD DAC evaluation criteria[2] of: Effectiveness, impact and sustainability, and will propose an outcome harvesting approach/method that demonstrates how the project has contributed to the outcomes.

The evaluation will combine remote desk review of existing relevant documents (incl. responses to surveys), group discussions and bilateral interviews of stakeholders, including APT staff directly involved in the project. Documents to review will be in English.  

The evaluator(s) will ensure confidentiality of the interviews and group discussions.

The evaluation method will be further developed in the evaluator’s inception report.

6. Budget and support from the APT

In addition to the budget of EUR 15’000 for the evaluation’s delivery (including evaluator’s fees and the possibility for field mission costs), the APT commits to provide the evaluator with all relevant documents and time necessary for the evaluation. This includes, inter alia:

• core project-related and APT strategic documents;

• any other relevant documents that have been developed as deliverables under the project, or that can help understand the project and its institutional framework;

• the list of persons and their contact details who have been involved in the project and could be selected for being interviewed. The APT will make sure they all understand and give consent on their participation to the evaluation process.

The APT has identified the following staff to support the evaluator:

• Regional Project Manager and Fundraising Manager: mainly to share and ensure access to all necessary resources; ensure interviewees’ information and consent; follow up on the deliverables and different steps of the evaluation process;

• Other APT staff team as relevant.

   

The APT will draft the contract, of which these Terms of Reference (ToR) form integral part.

7. Expected deliverables and timeframe

The following outputs are expected to be delivered by the evaluator as part of the evaluation process:

8. Qualifications of the evaluators

The consultant(s) should have the following qualifications and competences: 

• Proven record in conducting evaluations in the field of international human rights;

• Knowledge of, and experience in applying standard evaluation principles, qualitative and quantitative evaluation methods; experience of outcome harvesting method an asset;

• Ability to write concise evaluation reports of high quality in English;

• Knowledge about or experience of anti-torture work, especially in Asia;

• Independence and absence of conflict of interests by not having been in any way involved in any stage of APT programme design, implementation or monitoring.

9. Submission of bids and process

Applicants shall submit the following documents by Wednesday 15 January 2025 CoB, to jobs@apt.ch with “Evaluation Safe in Custody” in the title of their email:

• Cover letter (max. 1 page) demonstrating how the consultant(s) previous evaluation experiences of comparable projects but also their knowledge and expertise fit well with this assignment; 

• CV, including references to licences, certifications, accreditations, etc. 

• 2 final reports at least, of comparable evaluations carried out within the last 3 years;

• Contact details of 2 references expected to support claims of knowledge, skills and experience; 

• Proposed budget (outline is sufficient at the application stage).

For any specific questions about the evaluation, please contact Cécile trochu Grasso, Fundraising Manager, ctrochu-grasso@apt.ch. Questions after January 10, 2025 will not be answered. 

The selection of the consultant(s) will be decided during the week of 20 January 2025, with the contract expected to be signed the following week, aiming to commence the consultancy no later than 1 February 2025.