Request For Proposals: Media Program Evaluation Consultancy Services

Facebook logoTwitter logo

INTRODUCTION

Since 2011, NRGI implemented different iterations of a capacity development program for Strengthening Media Oversight of the Extractive Sectors. The program was first piloted in Ghana and Uganda in 2011 and 2012. It was rigorously monitored all through the pilot phase and evaluated periodically. A comprehensive evaluation of the pilot phase covering training activities up to October 2012 was conducted. Separate externally-led country-level evaluations were carried out in Ghana in 2014 and Myanmar in 2014-2015.

NRGI intends to commission the second comprehensive multi-country evaluation to provide evidence of the results and outcomes of the program in Ghana, Myanmar, Tanzania, and Uganda. The period the evaluation will cover is January 1, 2013, to March 31, 2016, except where different timeframes are specified.

PURPOSE OF THE CONSULTANCY

  1. To evaluate the quality, quantity, and scope of coverage of oil, gas, and mining issues in the target countries, including the degree of alignment with policy issues captured in the Natural Resource Charter (2nd edition)
  2. To evaluate the overall impact of the program on the individual and institutional growth and capacities of the journalists and media organizations that have participated in the program
  3. To take stock of and account for the whole range of factors deemed responsible for the documented results and outcomes of the program
  4. To provide a list of recommendations to NRGI and partner implementing organizations to improve the program and evaluative approaches


TERMS OF REFERENCE (TOR)

NRGI seeks the services of a firm or organization as Media Evaluator with experience and contacts in at least two of the target countries – or with demonstrated capacity to conduct multi-country media evaluations even if lacking experience in the target countries. The Media Evaluator will lead in designing and conducting the assessment across all four countries. Based on a work plan to be agreed with NRGI, the Media Evaluator will:

  1. Review and adapt NRGI's existing media monitoring and evaluation framework into a methodology that is fit for the purpose.
  2. Assess the coverage of oil, gas and mining issues in terms of:
    (a) Scope of reporting: extractive subjects covered, as captured by the policy issues considered in the Natural Resource Charter, using a template provided by NRGI (only for 2015 – see below);
    (b) Quality of stories, using and adapting as needed NRGI's story evaluation criteria scorecard (for all years);
    (c) Quantity of stories, using desk review of stories provided by trainees, NRGI, local partners and available online (for all years);
    (d) Media platform used (for all years);
    (e) Patterns, trends and changes in coverage of extractives governance over time on all the above dimensions; This means:
    i. For each country, establishing variations in the scope of reporting, quantity, quality and platform used for stories produced by NRGI trainees across data captures for each media class and across media classes;
    ii. For each country, establishing variations in the scope of reporting, quantity, quality and platform used for stories produced by NRGI target media houses across data captures for each media class and across media classes;
    iii. For each country, establishing variations in the scope of reporting, quantity quality and platform used for stories between the stories produced by journalists and media houses targeted by NRGI and those that have not been targeted (control group);
    iv. Across countries, establish variations in the scope of reporting, quantity and quality of stories between the stories produced by journalists and media houses targeted by NRGI, with a view to providing a cross-country comparative analysis on the effectiveness and impact of country programs and any related lessons).
  3. Track, document and maintain a record of the stories published or broadcast by the participating and non-participating journalists and media outlets.
  4. Trace the career development of the journalists who have participated in the program from the beginning.
  5. Provide case studies (three per country) of journalism on extractives by the journalists and media outlets that have participated in the class of 2015 in Ghana, Tanzania and Uganda.
  6. Prepare and submit reports of the evaluation plan and the findings with summaries, analyses, interpretations and recommendations based on the evidence collected.


SCOPE OF THE EVALUATION

For the purpose of bidding for this project, we provide the following additional background information on the scope of the evaluation:

  1. The evaluation will involve undertaking original data collection and analysis and comparing – as needed – with data and analysis from existing evaluations as follows:
  2. (a) Ghana
    § Collection and analysis of data for 1 target group of 8 trainees and media houses of origin for 2015;
    § Use of data and analysis from an existing evaluation for 2014;
    § Collection and analysis of data for 1 target group of 9 trainees and media houses of origin for 2013;
    § Use of data and analysis from an existing evaluation for 2010 to 2012.
    (b) Uganda
    § Collection and analysis of data for 1 target group of 8 trainees and media houses of origin for 2015;
    § Collection and analysis of data for 1 target group of 10 trainees and media houses of origin for 2014;
    § Collection and analysis of data for 1 target group of 9 trainees and media houses of origin for 2013;
    § Use of data and analysis from an existing evaluation for 2010 to 2012.
    (c) Tanzania
    § Collection and analysis of data for 1 target group of 8 trainees and media houses of origin for 2015;
    § Collection and analysis of data for 1 target group of 10 trainees and media houses of origin for 2014;
    § Collection and analysis of data for 1 target group of 10 trainees and media houses of origin for 2013 (there was no program prior to 2013).
    (d) Myanmar:
    § Use of data and analysis from an existing evaluation for 2014 (there was no program prior to and after 2014).
  3. Data captures: For original data collection, the data will be organized and analysed by capture points:
  4. (a) Capture I (baseline) will assess stories produced generally within 3 months prior to the target groups receiving training;
    (b) Capture II (during training) will assess stories published after the start of training;
    (c) Capture III (end line) will assess stories produced generally within 3 months after the training.
  5. Anticipated number of stories: We cannot anticipate how many stories each target journalist will produce or have produced for those years where original data collection is needed. However, from previous evaluations, we can estimate the following:
  6. (a) Number of stories by target journalists: In Uganda, the average number of stories produced by journalists collectively is 100 stories per class. In Ghana, the average is 56. In Myanmar, the data from the only evaluation conducted is 28. No data are available for Tanzania.
    (b) Number of stories produced by media target houses: In Uganda, the average number of stories produced by media houses collectively is 174 stories. In Ghana, the average is 312. No data are available for Myanmar and Tanzania.
    (c) Number of stories by control groups: Data from previous evaluations are only available for Myanmar where the control group produced 98 stories.
  7. Sampling: We expect the evaluators to determine the significance of sample sizes when undertaking the analysis of scope and quality of stories by journalists and media houses, for target and control groups. In previous evaluations conducted in Ghana and Uganda, the evaluators set at 30% the significance threshold for target journalists and media houses.


DELIVERABLES

  1. Inception Report and Methodology to be delivered by November 30, 2015. This report will include:
  2. (a) The Consultant's understanding of the TOR and relevant feedback to NRGI;
    (b) A clear rationale for the evaluation;
    (c) A comprehensive description and explanation of the methodology and instrument(s) to be used;
    (d) The approach to analysis and reporting of findings;
    (e) A management plan for the assignment with names, credentials and contacts of the country-level evaluators;
    (f) A detailed work plan for delivering the assignment from start to end;
    (g) An outline of the final report, showing how the data and findings will be organized and presented.
  3. Interim Report to be delivered by February 29, 2016. In addition to the standard content expected in an evaluation study, this report will include:
  4. (a) Relevant contextual and background information;
    (b) Detailed elaboration of the final methodology adopted;
    (c) The approach followed in conducting the study;
    (d) Comprehensive summaries of the findings;
    (e) Detailed analyses, interpretations and discussion of the findings;
    (f) Conclusions and recommendations.
  5. Draft Final Report to be delivered by April 15, 2016. In addition to all the elements specified for the Interim Report, this report will incorporate as appropriate the feedback provided by NRGI as well as its partners and stakeholders.
  6. Prepare and present the Draft Final Report in person at the NRGI Media Program Conference in May 2016.
  7. Final Report to be delivered by May 31, 2016. This report will incorporate as appropriate the feedback provided by NRGI as well as its partners and stakeholders, including updates, revisions and recommendations that will arise from the media conference.


NRGI'S ROLE

  1. Brief the Consultant about the media program.
  2. Avail all necessary information, documents, and materials.
  3. Provide technical expertise about the extractive industry to support the development of an effective methodology.
  4. Assist in identifying country-level evaluators.
  5. Review and approve reports and provide feedback as and when sought or required.


CONTRACT DURATION AND TERMS

The Consultant will be engaged over the period starting on or about October 30, 2015 to May 31, 2016. There is little flexibility in the timing of the evaluation and the submission of deliverables.

Payments will be made in instalments upon submission of satisfactory deliverables as follows:

  1. Inception Report and Methodology – 40%
  2. Interim Report – 30%
  3. Final Report – 30%


QUALIFICATIONS AND EXPERIENCE REQUIRED

  • Access to a research support system.
  • Experience in monitoring and evaluating media performance and training projects.
  • Proven competence in quantitative and qualitative methods of data collection, analysis and presentation.
  • Ability to present numerical and qualitative research in rigorous style and compelling narratives.
  • Good understanding of journalistic practices.
  • Familiarity with governance issues in the extractive sectors of oil, gas and mining.
  • Strong reporting writing skills.


APPLICATION PROCEDURE

  • Application deadline: October 2, 2015
  • Send the following electronically as PDF files where possible:
  • (i) An expression of interest including: (a) description of the firm or organization's competency and suitability for this assignment; (b) names, qualifications, and experience of the leader and key technical staff who will handle the assignment; and (c) names and contact details of three references of clients that NRGI will contact only if shortlisted.
    (ii) An estimated budget including a detailed breakdown of costs, fees, and applicable taxes.
    (iii) Two samples of similar work (at least one from any of the target countries if applicable).
    (iv) An up-to-date profile of the firm or organization.
  • E-mail the application and requirements to training@resourcegovernance.org
  • Use the following as the email subject line: Media Evaluation Consultancy Services
  • Note that NRGI will contact only shortlisted firms/organizations. No phone calls please.