Pivoting in the face of the unknown: Five adaptations for effective phone-based data collection during COVID-19

As originally published by Pathfinder International on September 1, 2020

Distance learning. Workplaces gone virtual or closed down completely. A dramatic global economic slowdown and travel at a near standstill. Such is the global reality in the time of COVID-19, where virtually every group of people is reimagining what were once routine activities—and researchers are no different.

For the (re)solve consortium that is working to support adolescent girls’ sexual and reproductive health (SRH) in urban areas of Burkina Faso, COVID-19 hit just as the project was entering its final evaluation phase. The overall evaluation—which aimed to assess the effectiveness of a school-based sexual and reproductive health (SRH) intervention—was planned for March 2020. But as COVID-19 took center stage in the global arena, Burkina Faso was one of the first and hardest hit countries in West Africa. Schools shuttered, curfews were imposed, and families sheltered in place.

As we took stock of the situation, we discussed several options, including delaying the endline until we thought it might be safe to do in person. But this option grew increasingly unlikely given the intensity of the pandemic and the anticipated difficulty of finding schoolgirl participants later, given summer holidays.

We raised the idea of conducting the endline evaluation entirely over the phone, but we were skeptical: call teenage girls on the phone while they are stuck at home during a pandemic and ask them about their sex lives? How would we ensure privacy? What if participants’ parents listened in, and how might this affect responses? Would cellphones hold a charge and connectivity last for the entirety of the call?

As we weighed these concerns, we knew that any option needed to, first and foremost, ensure that we were not introducing harm into the girls’ lives. But as we talked with colleagues and ran through various scenarios that might unfold, we concluded that we could proceed with phone-based interviewing while ensuring the utmost safety of our participants.

Adapting research methods for phone-based data collection

Others before us have treaded into the realm of phone-based data collection. We borrowed from these pioneers, whose tips and tricks of the trade came in handy. However, our research context was uniquely challenging: we had to locate adolescent girls during summer holiday in the middle of a pandemic, when many of them were dispersed across family households, as well as get parental approval for phone-based data collection. After jumping those hurdles, we had to ask girls about their sex lives and contraceptive experience over the phone – all while maintaining their privacy and discretion.

To move forward, we had to find the right combination of practices that (a) minimized participant burden and kept girls safe, (b) were logistically and financially feasible and (c) maintained high-quality data. In the end, we made several adaptations for our fieldwork that worked for us—and that could prove beneficial in other contexts:

1. Invest additional time during data collection training and piloting – it pays off: Our biggest concerns going into phone-based data collection related to rapport-building and maintaining data quality, privacy and mobile connectivity. We piloted phone-based interviewing with nearly 40 girls, during which interviewers assessed these aspects and practiced techniques we had developed to mitigate these concerns. Interviewers asked participants to give feedback, especially related to their comfort answering sensitive questions on the phone. One tactic we piloted that worked well was to prompt girls to inform interviewers when someone had come into the room by changing the topic to the weather or their school exams. Interviewers were also trained to pause at the beginning of each section and make sure the conversation was still private.
2. Obtain parental approval/consent again, if needed: For our endline, we decided to renew approval from parents – even though we had obtained their consent at the beginning of the study. As we shifted methods to remote interviewing and several girls moved into different family living situations, we thought it best to obtain approval (either from the parent who initially provided consent or from new guardians) over the phone. This was certainly an extra logistical step, but we believed that having parents’ knowledge about their daughters’ participation was essential in this transformed context.
3. Shorten survey tools (as painful as that might be): The process of reducing survey length is a painful process for any researcher to make given the seeming myriad insights to be unlocked from each question. But given the potential for phone interviews to double in length due to shifting privacy and connectivity issues, we had to substantially trim our quantitative and qualitative tools. As daunting as the task seemed, we were surprised to realize that the revised survey tools more efficiently captured the information we needed for our analyses – a practice we will take forward for future projects.
4. Design a ‘structured pause’ to check data quality: With phone-based interviewing, we worried that girls might give false answers or not properly answer a question, either out of discomfort or due to privacy concerns. To assess this, we paused after the team had collected roughly 10 percent of the quantitative sample to conduct several assessments of the data – primarily comparing the preliminary data to baseline data and assessing emerging trends on our main variables of interest.
5. Use the opportunity to learn how COVID-19 has affected communities: To take the pulse of the situation in Burkina, as well as contextualize our evaluation findings, we added three questions on how COVID-19 was impacting girls’ lives, specifically in ways that might impact their ability to go to the clinic or think about sex and contraception.

What we learned

Overall, we were pleasantly surprised with the feasibility of data collection by phone with adolescent girls in Burkina Faso. At the 10 percent checkpoint, we had good indication that the data was of high quality, and that girls were participating. These initial indicators held true, and throughout data collection, parents continued to give permission for their daughters to participate. Refusals by parents or girls were few and far between. The data collection team reported that girls were answering questions seriously and within the estimated, Institutional Review Board (IRB)-approved time (approximately 45 minutes). Although the team struggled with connectivity issues—mainly dropped calls—they were able to reconnect and finish the interview in the same window of time. In the end, our dynamic data collectors successfully gathered survey data from 2,081 girls (88 percent completion rate) and conducted 41 in-depth interviews.

The COVID-19 pandemic and corresponding lockdown have interrupted and transformed lives around the world, presenting challenges we could never have foreseen or imagined. Like many researchers in 2020, we have had to give up on “the perfect plan” and learn to adapt, while keeping our research participants safe. Working on an iterative project gave us the foundation to come together, test new approaches, and find alternatives that work—in a COVID-19 world and beyond.

About (re)solve: The (re)solve project is a collaboration between Pathfinder International, Ideas42, Camber Collective, and International Center for Research on Women that aims to increase contraceptive use among women and girls in the developing world. In Burkina Faso, (re)solve consortia partners implemented an interactive, facilitated board game with adolescent girls in Ouagadougou and Bobo-Dioulasso from November 2019 to March 2020. The main objectives of the game were to help girls understand relevant sexual and reproductive health information, such as the use of contraception, and facilitate a follow-up visit at a nearby clinic for more information. Game facilitators gave participants “passports” that could be used at participating clinics for discrete sexual and reproductive health services.

About the (re)solve evaluation in Burkina Faso: The main objective of the evaluation was to measure the effectiveness of the intervention on key outcomes such as intention to use family planning and follow up at a health center for sexual and reproductive health information and services. To reach this objective, we used a pretest/posttest RCT study design to evaluate the effectiveness of the (re)solve intervention. We collected data at baseline from a random selection of 2,323 girls at both intervention and control schools in 3^rd and 4^th grade (equivalence of 9^th and 10^th grade in the US) from December 2019 through January 2020. We interviewed girls in the intervention schools at midline—within approximately one week of playing the game, between January and March 2020—and our endline aimed to follow up with the same girls interviewed at baseline approximately three months after playing the game. At endline we also conducted key informant interviews with game facilitators, clinic staff and key stakeholders (with e.g. school administrators, government officials). Final evaluation results are forthcoming.