How Samaritan’s Purse Accomplished an Internal Program Evaluation
with the Help of Elite Research.

Elite Research, from the very beginning, has been a valuable partner and resource for our Program Evaluation project. They provided the expertise, academic resources and technical support necessary. In a non-technical environment, they speak our language.

Michael Cardy, Strategic Initiatives Manager Samaritan’s Purse, Operation Christmas Child

In 2014, Elite Research began working with Samaritan’s Purse’s (SP) to conduct an internal study of their Operation Christmas Child (SP-OCC) program, a follow-up faith-based program. Their desire was to evaluate the immediate, as well as the long-term impact of the program, and requested the evaluation include both quantitative and qualitative approaches. Conducting such a study would assist Samaritan’s Purse in understanding the global impact of its SP-OCC program upon its participants, as well as their teachers, communities, and churches. It was intended that this study become a benchmark for future evaluations.

With the evaluation goals of assessing program impact on content comprehension, behavioral change, and transformation (child, teachers, community, and churches), Elite Research worked with SP-OCC leadership to develop a survey instrument that considered program materials and delivery as its foundation and a thorough methodology that could be easily replicated. The methodology included an international pilot project that tested the survey and procedures in four distinct (but representative) countries, using a pre-test/post-test design with control and experimental groups. The pilot study allowed for testing of comprehension, language, data entry and preliminary analysis, along with initial scoring to identify any problems with questions. Towards the end of this 6 month pilot phase, an international SP-OCC team was trained in the necessary procedures to roll out the global study (in 3 countries in 2014 and 4 countries in 2015). Additionally, a scoring index was created based on the construct of questions, which would allow for a very quick score on a spirituality scale. This scale provided a more efficient benchmark assessment, as instead of evaluating specific questions, it could evaluate specific constructs or categories of thought.

Once the quantitative pilot study was well on its way, the qualitative study began. Its particular focus was to gain a rich perspective from participants, teachers, community members, and church leaders regarding the effectiveness of the program. Focus groups and interviews were used to gain in-depth data to answer key questions posed by Samaritan’s Purse. Elite Research worked with SP-OCC leadership to develop key questions and the focus group design (groups, format, facilitation scripts, data analysis training, and reporting). As part of this phase, 21 focus groups were conducted in 4 countries.

Having both quantitative and qualitative data enabled Samaritan’s Purse to have a comprehensive picture of the impact of the SP-OCC program. As a result of this program evaluation, Samaritan’s Purse has been able to…

  • Define and measure the impact of the SP-OCC program within the surveyed countries.
  • Reliably generalize those findings – when combined with other internal reports – to gauge the spiritual impact.
  • Gain valuable insight to inform management decisions.
  • SP-OCC Methodology

    • Mixed method design
    • Minimum required quantitative sample of 500 per country
      • Pre-test/post-test
      • Control/experimental groups
    • Minimum required qualitative sample of 64 per country
      • Children
      • Teachers
      • Community leaders
      • Church leaders
    • Train-the-trainers
     

    The key things for the client on this project were low cost and embedding skills into their teams to be able to replicate future studies. Additionally, Samaritan’s Purse’s strict data privacy requirements, did not allow data to be handled outside of their offices; Elite Research worked with this stipulations by providing step-by-step, detailed procedures for data collection and analysis, and by providing 2-day onsite analysis training and oversight key analysis steps.

    About Samaritan’s Purse.

    Samaritan’s Purse (www.samaritanspurse.org) is a nondenominational evangelical Christian organization providing spiritual and physical aid to hurting people around the world. Since 1970, Samaritan’s Purse has helped meet needs of people who are victims of war, poverty, natural disasters, disease, and famine with the purpose of sharing God’s love through His Son, Jesus Christ. The organization serves the church worldwide to promote the Gospel of the Lord Jesus Christ.

    About Elite Research, LLC.

    Elite Research, LLC is a woman-owned, small business providing research and statistical consulting with headquarters based out of Dallas, Texas. As a global provider of research and statistical consulting, we provide expert support to academic researchers, government entities, nonprofit organizations, researchers in the private sector, and corporations in all facets of the research process, including the sampling, research design, data collection, analysis, reporting, and implementation and optimization of findings. With over thirty consultants from a wide spectrum of professional and academic backgrounds, Elite Research has the ability to cater to almost any research or statistical need. Our ambition is to ensure that both research and statistics are easily approachable by everyday researchers globally.

    Our Nonprofit HomepageOur Nonprofit Services Our Nonprofit White papers