Mapping the Development Journey: Using large-scale surveys to understand context and address learning needs

This blog post was written by Ali Hayat (Chief of Party, MESP) and Dan Killian (Technical Manager, MSI).

Decision-makers in international development are increasingly inundated with information. Effectively navigating and utilizing this information is complicated by operational and technical challenges. The methodological differences between the available data streams make it difficult to meaningfully aggregate or disaggregate findings. The available information may also present conflicting accounts of the context, without sufficient explanations for understanding “why” this may be or to “what extent” certain findings may be generalizable, and “how” the different findings may fit together. Thus paradoxically, information excess results in a fragmented understanding of the overall context in which decision-makers operate.

Within large USAID missions, such challenges also make it difficult to holistically assess Mission performance and contributions while complicating any understanding of the relationship between donor interventions and a country’s overall development journey. All of this has a significant bearing on the ability of the decision-makers to make informed choices. As the largest donor in country, USAID/Jordan manages over 40 implementing partners and, with the five-year Memorandum of Understanding signed between the governments of the U.S. and Jordan in February 2018, a minimum annual budget of over $1.25 billion through FY 2022.

To enhance a better understanding of the overall context, USAID/Jordan requested its Monitoring, Evaluation and Support Project (MESP) to design a large-scale survey in order to measure mission indicators in the Jordanian population and among self-identified beneficiaries, provide implementing partners with survey data that could potentially be adapted for activity evaluation, and explore a learning agenda to identify and better understand the determinants of indicator performance.

Survey Development and Implementation

The MESP team started with a formal data and document review process, including documents and material on the Jordan Development Knowledge Management Portal (KAMP). This process was followed by semi-structured learning agenda discussions with four USAID technical teams, including Democracy and Governance (DRG), Economic Development and Energy (EDE), Education and Youth (EDY) and Water Resources and Environment (WRE). The purpose of these discussions was to understand the learning needs of technical teams that may be addressed through a large-scale survey. In addition to reviewing existing survey instruments, the MESP team also met with several implementing partners, and the larger US Embassy community.

The final version of the survey has modules on household information, general conditions and public services, education, citizen participation, women and society, employment, entrepreneurship and respondent background. Specifically, the survey covers six mission performance indicators from all of USAID/Jordan’s Development Objectives (DOs), while also measuring a selected number of activity performance indicators. Given the geographic spread of USAID in Jordan, a nationally representative sampling plan was developed with a survey sample size of 12,000. This will allow the survey to provide national and sub-national level context and trends, as well as the ability to disaggregate data based on key demographic variables of interest such as age, sex, income, nationality, and level of education.

Survey Utilization 

The survey results provide various utilization possibilities:

• By providing contextual data at the national and sub-national levels and corresponding to USAID’s DOs, indicators, themes and learning needs, the survey data can inform USAID’s internal portfolio review conversations.
• The results from this survey will help answer critical questions of interest to the USAID technical teams to help improve ongoing activities as well as to more effectively design future activities.
• The survey can provide a national and sub-national level baseline of key indicators, which can be tracked over time, assuming the survey is repeated. In the case of Jordan, where a new CDCS may be developed soon, this survey could also provide a good baseline of public opinion.
• The survey findings can be used by both USAID and implementing partners as baselines for specific activities, as well as to estimate realistic targets by tracking change over time.
• The donor exposure questions allow for identifying the type and extent of exposure. Responses from these self-identified beneficiaries provide an opportunity to test theories of change, as well as to set up realistic targets.
• Findings from this survey can contribute to the current conversations related to self-reliance by providing tailored country-specific data on general population perceptions and opinions.
• Exploration of survey data may generate interesting questions that may be investigated more thoroughly by individual assessments or activity-level data collection.
• The findings from the survey will also be useful for a much wider group of stakeholders including USAID, USAID IPs, donors, Government of Jordan, private sector entities, and non-government organizations working in the development realm. 

To learn more, find a resource on this initiative and a survey instrument template here.