The Balkan Investigative Reporting Network (BIRN) is the leading independent non-governmental organisation in the Western Balkans, specialising in media development, investigative journalism, and public sector oversight. Established in 2004, BIRN operates as a network of NGOs dedicated to promoting freedom of expression, human rights, and democratic values across South-Eastern and Central Europe.
BIRN is active in Albania, Bosnia and Herzegovina, Kosovo, North Macedonia, Romania, and Serbia, and maintains an editorial presence in Greece, Bulgaria, Croatia, Moldova, Montenegro, Ukraine, the Czech Republic, Poland, Slovakia, Hungary, and Slovenia.
BIRN Kosovo is the network’s largest member, with 47 staff members. Founded in 2005, its core mission is to inform citizens and support Kosovo’s democratic transition by promoting accountability, the rule of law, and policy reform through professional investigative journalism. In October 2013, BIRN Kosovo and BIRN HUB partnered in what became the first phase of the Paper Trail to Better Governance which is currently in the fourth phase of implementation supported by Austrian Development Agency (ADA) since its initial phase in 2012.
The primary objective of the project throughout all phases of implementation is to advance good governance, transparency, accountability, and the responsiveness of public institutions to freedom of information in the Western Balkans. In 2023 BIRN Kosovo, in partnership with BIRN HUB, secured EUR 1,007,000 in funding from the ADA for a three-year project (Phase IV implementing period September 2023-August 2026) aimed at building on the impact, achievements, and lessons learned from the three phases of BIRN’s “A Paper Trail to Better Governance” initiative. The Paper Trail to Better Governance IV project is a cross-regional initiative that supports the European Commission’s membership requirements by promoting the rule of law, accountable governance, and an independent judiciary in the Western Balkans.
Purpose and objectives of the mid-term evaluation
BIRN is commissioning an external midterm evaluation of its Action. Conducted for both BIRN and the Austrian Development Agency (ADA), this evaluation aims to assess progress to date and guide the continued implementation of the Action. It will offer an independent analysis of current achievements, ongoing challenges, and lessons learned so far.
The evaluation’s primary purpose is learning and as such to deliver evidence-based findings, conclusions and recommendations to help refine BIRN’s programs, strategies, and activities during the remainder of the implementation period. Additionally, the evaluation will provide evidence to inform decisions on the potential future continuation and/or scaling up of the initiative.
The evaluation will focus on following primary objectives:
- To assess the project’s relevance, effectiveness, impact and prospects for sustainability
- To provide practical, evidence-based recommendations that will help enhance ongoing and future implementation and decision making at strategic and operational level.
The main user of the evaluation results will be BIRN. The insights generated are expected to strengthen BIRN’s impact, improve its operations, and inform its strategic direction beyond the current project. They are expected to be also relevant for other civil society organizations, donors, and development partners pursuing similar objectives in transparency, accountability, and good governance.
Scope of the evaluation
This external evaluation shall be carried in accordance with the ADA Guidelines for Programme and Project Evaluations. It is conducted during the second year of the three-year project and will generally cover the time period from September 2023, the start of phase IV of the project to the end of the data collection for this evaluation. For the assessment of impact, the evaluation will, however, also rely on evidence generated during previous phases of implementation.
In terms of geographic scope, it will encompass implementation in Kosovo and Bosnia and Herzegovina as well as all the Western Balkan countries (Albania, Montenegro, North Macedonia and Serbia). While data collection in the field will be conducted primarily in Pristina, Kosovo and Sarajevo, Bosnia and Herzegovina, the engagement with stakeholders across the broader Western Balkan region will be conducted remotely.
The evaluation will cover all components of the project. It will assess the types of support, approaches and strategies applied and their role in results’ achievement and identify related lessons learned. The evaluation team is expected to review the project’s intervention logic and if needed suggest improvements.
Given that ADA has supported BIRN through several interconnected projects aimed at achieving long-term goals, this evaluation presents a unique opportunity to partially assess the broader impact of that support, beyond the results of individual project phases.
Evaluation Questions
Relevance
- To what extent has the project’s relevance changed over the implementation period, in particular with a view to changes in the context (including, but not limited to recent US funding cuts)?;
- How has the project adapted its approach or priorities in response to the changing context?
Effectiveness
- How has the project contributed to training and empowering its target groups, particularly journalists and to what extent have the trained journaliss
- applied their acquired skills within their workplaces?
- How have other media outlets in the region engaged with BIRN’s capacity building efforts and with what results?
- What were the key factors — both positive and negative — that influenced the achievement or non-achievement of the project’s objectives?
Impact
- To what extent has the project contributed to reducing corruption and increasing transparency in the Western Balkans?
- What have been hindering and facilitating factors for impact?
9. Sustainability
What is the likelihood that results generated by the project will continue after its completion and what have been hindering and facilitating factors for sustainability (internal and external)?
Design and Approach
The evaluation shall apply a mixed-methods approach, combining:
- Document review
- Semi-structured interviews and focus group discussions (in Kosovo and Bosnia and Herzegovina in-person; online with representatives based in other countries.
Data collection, analysis and reporting must Gender-sensitive and inclusive data disaggregation.
The evaluation will proceed in several phases, aimed at collecting data and answering the evaluation questions outlined in Chapter 4:
- Contracting and Kick-off Meeting: The kick-off meeting will take place only after the contract between BIRN and the selected bidder has been signed. Prior to the meeting, project documentation, annexes, and reports submitted to ADA will be shared with the evaluation team to allow for an initial review. The meeting will then serve to clarify any outstanding issues and finalize roles as needed.
- Desk Research and Documentation Analysis: The Evaluation Team will review all relevant existing data, including but not limited to project documentation
- Preliminary interviews: preliminary interviews can be held with n introductory (hybrid) meeting with the project and editorial teams will focus on project management, coordination, and available resources.
- Inception Report: The Evaluation Team will prepare an Inception Report outlining the detailed evaluation design, methods, tools, work plan, deliverables, timelines, and mitigation strategies, following the structure in Annex 5 (“Quality Checklist for Inception Reports”). As part of the Inception Report, the Evaluation Team will draft an Evaluation Matrix, aligned with Annex 7 of the ADA PP Evaluation Guidelines, see ibid, p. 51).
- Field Visits and Hybrid Meetings: In-person interviews and focus groups will be conducted in Kosovo and Bosnia and Herzegovina, with online consultations for stakeholders based in the wider region and beyond.
- Debriefing and Presentation of preliminary findings: After the end of the data collection, the evaluation team will provide an overview of the data collection and stakeholder inclusion and present the preliminary findings conclusions as well as preliminary recommendations to BIRN and ADA (online/hybrid meeting).
- Evaluation report (draft and final): The draft evaluation report will be submitted to BIRN for formal written feedback (BIRN, ADA), finalized by the evaluation team after consideration of the feedback and approved by BIRN.
- Presentation of the Final Evaluation Report: The final evaluation report will be presented to an audience selected by BIRN and ADA, in a hybrid format.
Proposed methods:
For the different phases of the evaluation, the Evaluation Team will collect data and information using a mix of qualitative and quantitative methods, including:
- Desk research and document analysis of key project materials, reports, and annexes previously submitted to ADA and BIRN;
- Structured and semi-structured interviews conducted through face-to-face or online meetings with approximately 10 stakeholders, including project staff, implementing partners, and institutional stakeholders. These interviews will take place across three primary locations: Prishtina, Sarajevo, Tirana. [Location A], [Location B], and [Location C];
- At least one focus group discussion (FGD) with 8 youth participants who completed the internship programme, ideally conducted in-person in Prishtina, Kosovo or online if
This approach will inform the estimation of working days required for primary data collection in the workplan.
All data collected needs to be disaggregated by sex, ethnicity and where feasible also in terms of disability. It is expected that the Evaluation Team will present concrete recommendations which are addressed to the specific stakeholders. In addition, all research questions should be addressed paying attention to gender, social and environmental issues.
The Evaluation Team needs to base its work and methods in line with the Guidelines for Project and Programme Evaluations developed by the Austrian Development Agency and OECD DAC norms need to be considered throughout the entire evaluation process (found at: https://www.entwicklung.at/fileadmin/user_upload/Dokumente/Evaluierung/Evaluierungs_Leitfaeden/Guidelines_for_Programme_and_Project_Evaluations_ADA_2020.pdf).
Expected outputs and deliverables/Report(s)
Inception Report
- Inception report (approx. 10-15 pages) containing appropriate methodology to be applied during the mid-term evaluation, as well as the work plan to be used during the course of the assignment is drafted, submitted, and approved by BIRN. Report must be in line with the Annex 5: Quality Checklist for Inception Report (IR), 46 and 47;
Evaluation Report
- The evaluation report (about 25-30 pages without annexes) including executive summary, findings and recommendations which must be in line with the Guidelines “Quality Checklist for Evaluation Report (ER)” 48-50;
- Filled out Results Assessment Form as integral part of reporting in line with Annex 9 Results Assessment Form (RAF), p. 53;
All reports need to be written in English.
Workplan
The assignment is estimated to require 79 working days. The midterm evaluation is expected to begin in June and be completed by the end of August 2025. The first draft report must be submitted by September 15, 2025, with the final report due by September 30, 2025.
Activity | Expert 1 / Team Leader | Expert 2 / Team member | Total days per activity |
Desk research | 6 | 4 | 10 |
Draft and finalize inception report | 7 | 3 | 10 |
Field visit (BiH and Kosovo) | 12 | 12 | 24 |
Virtual data collection | 5 | 3 | 8 |
Data analysis | 4 | 3 | 7 |
Draft and finalize evaluation report | 10 | 4 | 14 |
Debriefing / Presentation | 3 | 3 | 6 |
Total | 47 | 32 | 79 |
The written feedback from BIRN Kosovo, partners and the donors is planned to be given within a maximum of two weeks after submission of the reports.
Evaluation Management Arrangements
The BIRN Project Manager will serve as the main point of contact for the Evaluation Team, facilitating communication, providing project documents, and assisting with stakeholder engagement and logistical arrangements.
The Evaluation Team will operate independently from the project and organizational management structures of BIRN Kosovo and BIRN HUB.
BIRN’s management will fully respect and uphold the Evaluation Team’s independence, impartiality, and ethical standards, in accordance with established evaluation principles and the BIRN Code of Ethics.
Requirements for the Company/Team of Evaluators
BIRN is seeking a team of two evaluators or a company proposing at least two experts — a Team Leader and a Team Member — with at least one team member being a woman.
Candidates who were involved in any stage of the project’s design or implementation will not be considered.
Required Experience and Qualifications for companies and/or individuals within a company or Evaluation Team:
- Proven experience in conducting and leading evaluations, preferably in the fields of good governance, rule of law, or media sector;
- Experience in evaluating regional projects is an asset;
- Excellent communication skills in English; proficiency in Albanian or Serbo-Croatian is mandatory for at least one team member;
- Strong communication, analysis, and presentation skills;
- Submission of a professional profile for the evaluation team/company (including CVs of all team members);
- At least one team member must have experience in projects involving gender mainstreaming and/or environmental protection;
- Submission of at least two samples of previous evaluations conducted by the company or members of the proposed Evaluation Team.
Specific Requirements for the Team Leader/Senior Expert:
- At least 7 years of overall professional experience with a Master’s degree in a relevant field;
- Minimum of 5 years’ experience working with regional or international development projects and conducting evaluations using OECD/DAC criteria;
- Lead role in at least three evaluations, preferably with experience in the Western Balkans;
- Experience in program formulation, monitoring and evaluation, and qualitative data collection and analysis;
- Proven ability to produce professional, concise, and reader-friendly evaluation reports;
- Fluency in English (written and spoken) and good knowledge of Serbo-Croatian or Albanian.
Specific Requirements for the Team Member:
- At least 7 years of professional experience and studies in Law, Political Science, International Development, or the Media sector;
- Previous experience working on rule of law or media-related projects;
- Experience in qualitative data collection, analysis, and interpretation;
- Solid understanding of results-based and impact evaluations;
- Fluency in English (written and spoken) and good knowledge of Serbo-Croatian or Albanian.
Specifications for the Submission of Offers Application process
2Qualified and interested candidates are asked to submit an electronic copy of their expression of interest/proposal by June 2 2025 at 23:59pm, with the subject REF: “Paper Trail for Better Governance IV – Evaluation” to [email protected] and [email protected]
Applicants must submit the following:
Technical Proposal:
- Suitability Proof/Document:
- For companies: A profile demonstrating suitability for the assignment based on previous work experience and qualifications, including short biographies of experts and attached CVs (maximum 5 pages).
- For individuals: A team profile outlining relevant previous experience aligned with this ToR and detailed CVs for all members.
● Team Composition:
- Clear assignment of roles between the Team Leader and Team Member, ensuring that at least one member is a woman.
● Methodology:
- A proposed approach for carrying out the evaluation, including data collection and analysis methods, sampling strategies, triangulation, risks and mitigation
● Draft Work and Timetable:
- A suggested work plan and timeline, fitting within the number of working days proposed in this ToR.
Financial Offer:
- A detailed financial proposal, broken down by deliverables and individual daily
- The budget must be all-inclusive, covering accommodation, transportation, and any other related expenses, if applicable.
Evaluation of Offers:
The Technical Proposal will be evaluated with 70% whereas the financial one will be evaluated at 30%. The financial offer will be evaluated by using the best value for money approach (combined scoring method).
When using this weighted scoring method, the award of the contract should be made to the company or Evaluation Team, whose offer has been evaluated and determined as:
- Responsive/compliant/acceptable, and
- Having received the highest score out of a predetermined set of weighted technical and financial criteria as explained below:
- Technical Criteria weight; [70 points]
- Financial Criteria weight; [30 points]
Criteria | Weight Max. | Point |
Technical evaluation: | 70% | 70 |
● Relevant professional experience in line with Evaluation Requirements; | 25% | 25 |
● Quality of prepared Methodology and methods to be used in line with the Evaluation Requirements | 35% | 35 |
● Team Composition, gender and previous experience in thematic and geographical areas as required in the ToR | 10% | 10 |
Financial offer: | 30% | 30 |
Only candidates obtaining a minimum of 49 points would be considered for the Financial Evaluation.
BIRN has a maximum budget of EUR 30,000 for carrying out the required Mid-Term Evaluation.
ANNEXES
The Evaluation Team should take note of the following documents, comprising annexes to this ToR:
Annex 1 – Short Project Summary
Annex 2 – Paper Trail Investigations – Focus Page with investigations produced by the project
Annex 3 – BIRD Source Database
Annex 4 – ADA Guidelines for programme and project evaluations
Annex 5 – ADA Results Assessment Form (RAF)