Ma3an’s social network analysis: a wellspring of CLA
Measuring Collaboration
For programs like USAID Ma3an, a recurring conundrum is how to measure concepts that are core to civil society projects. How do you quantify collaboration? How do measure resilience? FHI 360 staff had to tackle this difficulty head-on when they placed collaboration at the heart of their theory of change (TOC).
The solution came as a social network analysis (SNA). For the uninitiated, SNA is a quantitative analysis that reveals dynamics between autonomous agents. In its graphical form, it is a network, visualizing the depth, type, frequency and direction of relationships in a system. For its performance indicator ‘percentage of project beneficiaries reporting quality collaboration,’ Ma3an pooled SNA data on the frequency of interactions, diversity of interactions, and diversity of sectors to construct an index of collaboration.
Ma3an completed its baseline SNA in 2019, with plans to repeat the study in 2021 and 2023 to assess evolution of connections density and numbers. Although SNA originally responded to an evaluation need, the monitoring & evaluation (M&E) team saw its potential to inform programming. Over time, the SNA evolved to be a flagship case of utilization-focused evaluation, a philosophy that “begins with the premise that evaluations should be judged by their utility and actual use.”[1] What follows is an account of the SNA’s galvanizing role in bringing about a wave of collaborating, learning, and adapting (CLA).
1. Expanding the Technical Evidence Base to Adapt
When SNA results were shared, staff realized that SNA could enrich the technical evidence base. The analysis showed who was at the margins, which stakeholders carried influence, and how the relationships varied between groups. This information could be used to adjust program activities, justify design choices, and create new interventions.
The SNA highlighted a need for private sector engagement (PSE). The report showed that corporate entities were weakly linked to the networks and to each other, often remaining at the periphery of Ma3an’s target Tunisian communities. A critical insight was that Ma3an needed to work on corporate social responsibility (CSR); SNA results suggested more time and engagements efforts were needed to increase the depth and quality of interactions between the private sector and target communities.
The analysis produced actionable insights by revealing who were the so-called gatekeepers and isolates. Gatekeepers, also known as hubs, are well-connected individuals who bridge sub-communities and are said to have high betweenness centrality. “Both empirical and theoretical results indicate that hubs can have a quite disproportionate effect, playing a central role [in …] resilience, despite being few in number.”[2] Conversely, isolates are individuals with few interactions to the system who tend to be marginalized. Teams used these metrics to strategically select whom to work with and which relationships to build.
For example, Ma3an found that civil society organizations (CSOs) were far more connected to communities than initially supposed and pivoted to involving them as strategic connectors. The analysis also identified key movers. For their community resilience committees (CRCs)— task forces of some six-to-eight individuals representing communities—Ma3an used the SNA to complement selection criteria: instead of selecting people only based on their engagement, the program also considered how many people they could connect the community to.
The gender equality and social inclusion (GESI) team similarly embraced the trove of findings in the data. The analysis revealed assortative mixing by gender, i.e. that women mostly connected to other women. Furthermore, women seldom held decision-making roles. In contrast, male stakeholders collaborated more frequently and deeply. There were also more degrees of separation between women and given sectors in the network.
From this data, Ma3an’s GESI team made numerous adaptations. It mandated that its local partners work closely with influential women, particularly women who could bridge sectors with isolated female communities and companies, and set quotas on selection. Because the SNA revealed that Disabled Persons Organizations (DPOs) were hubs for reaching mothers, the team more aggressively targeted DPOs. “This readjustment on how we target was the big thing that the SNA did for us,” says Asma Ayari, the Ma3an GESI Advisor.
The SNA provided hard data on the quality of women’s social relations — data that are often lacking or merely exist anecdotally. Moving into the mature phases of the project, Ayari said she would closely follow new iterations of the SNA. “We want to see if age group homophily changed or not, if women from different sectors are connected, and if they are interacting more with men. If women in the private sector connected more [than at baseline], then the heightened collaboration could mean more job opportunities for other women.”
For some, the SNA baseline was a sobering reminder that learning does not always sync with planning. “Timing is everything,” says Slim Yaich, the Technical Advisor on Ma3an. Although the SNA produced rich technical insights, the findings came after many of Yaich’s interventions were in motion. “If possible, it would be great to do the future SNAs before our CSAPs, as this could inform stakeholder selection.”
2. Scenario Planning for Self-Reliance
In development circles, scalability and self-reliance are hot topics. To paraphrase Larry Cooley, the success of interventions ought to be judged less by what you can add, and more by what you can remove.
For the Ma3an sustainability team, the SNA allowed them to engage in exactly this type of thinking. Using baseline data, they looked at what would happen if project-funded partners were removed from networks. The experiment showed that certain communities were heavily dependent on Ma3an’s partners. Staff not only found that certain interventions would have weak links post-Ma3an, but could point to clusters of beneficiaries who were likely to struggle to be self-reliant.
Taha Yousfi, Ma3an’s Sustainability Advisor, saw that the SNA underscored the need for a self-reliance strategy that focused on creating enduring relationships. “Today, the Ma3an sustainability approach is not only about the activities or physical structures,” explains Yousfi. “It is also about the dynamics, about the collaborative approach, and sustaining the dialogue spaces.” Using the SNA data, Yousfi worked with staff and partners to plan exit strategies before launching initiatives. “We cannot work on sustainability later . The local partners will be out of the picture in a few months, and they hold all the cards— they connect all the dots,” says Yousfi. “Once the intervention ends, the local partners will not be as active as they used to be.”
Similar scenario analyses were done looking at government representatives within networks. SNA uncovered some important realizations for scalability. On first blush, the analysis suggested that some of the most connected agents were appointed individuals from government delegations. These individuals moved constantly between one community to the other and were great connectors. But a deeper analysis showed that as these individuals were appointed by government officials, their tenure in those stations would be short-lived. As a result, the team selected individuals who were well networked and likely to remain as lasting figures in the community.
Interventions scale when they are taken up by governments or markets in the long run, and the SNA gave Ma3an the tools to examine both. Using the SNA, Ma3an integrates sustainability discussions from the beginning. “We are working on a miniature version of an exit strategy with the local partners and community stakeholders to gradually transfer all the responsibilities from the local partners—the organization that is contracted by FHI 360—to the community members. It is a big win to have strong data to measure the transition,” says Yousfi. (See Box 1)
3. M&E for Learning and Theory of Change
Monitoring and Evaluation (M&E) practitioners will tell you that theory of change is as much a product as it is a process. Yet in a world preoccupied with deliverables and deadlines, the latter is often overlooked. What Ma3an found is that their SNA ushered a surge of TOC-as-a-process: With the SNA shared, staff proponents of the analysis proliferated, and soon everyone began discussing the implications of the findings on their interventions. As one MEL team member put it, it shifted the question to “are we doing things right” to “are we doing the right things?”
A data-driven culture followed after the SNA was presented. “When we started the SNA study, it was something strange for the team,” says Amel Hammami, the analyst leading the study. “Staff didn’t know what we meant and for them it was something complicated. But after [when they saw the sociograms], the whole team started talking about SNA” and using it in their work an planning. In this way, SNA drummed up enthusiasm in using M&E data for learning.
The SNA not only expanded the audience for theory-of-change discussions, it let teams consider theories in parallel. “The SNA gave us a million ways to look at things,” says Hammami. Staff began engaging in multi-track thinking, because different networks implied different needs. “You would think that you cannot take a tailored specific approach for each community,” says Yousfi. “But the SNA says different. You have to take the characteristics of each community into account.” If there had been any doubt, the SNA reinforced the idea for all that cookie cutter approaches would not work.
For example, in hyperconnected communities like rural Laaroussa, interventions could not look the same and have similar dynamics to disconnected urban community of Sousse Riadh. In the latter more of an effort needed to be made to bring people together and demonstrate the win-win aspects of collaboration.
A Boon for CLA
Executives are constantly looking for game changers: power moves that will transform the way business is done. In Ma3an’s case, social network analysis proved to be one such game changer. Starting as a mechanism for indicator reporting, the foray outgrew is initial purpose and became a lodestone for CLA. The adaptations from the SNA learning made Ma3an’s interventions more strategic, inclusive, and collaborative. Will Ma3an’s programs scale? Will the relationships built between government, private sector, and community actors endure? No one has a crystal ball, but SNA stimulated the long-term thinking to give Ma3an’s communities the best shot at becoming resilient and self-reliant.
Ma3an ("Together" in Arabic) is a 5 year project implemented by FHI360 and over 30 international and Tunisian NGOs. It aims to provide 33 of Tunisi'a most vulnerable communities with the tools and resrouces needed to build resilience in order for them to withstand political, social and economic stresses and shocks. #Ma3an_Tunisia[1] Patton MQ. Utilization-focused evaluation, 4th edition. Thousand Oaks, CA: Sage; 2008.
[2] Newman ME. Networks: an introduction. Oxford: Oxford UP; 2010.