Data Collection Smart and Simple: Evaluation and Metanalysis of Call Data From Studies Applying the 5Q Approach

Abstract

Agricultural development projects often struggle to show impact because they lack agile and cost-effective data collection tools and approaches. Due to the lack of real-time feedback data, they are not responsive to emerging opportunities during project implementation and often miss the needs of beneficiaries. This study evaluates the application of the 5Q approach (5Q). It shows findings from analyzing more than 37,000 call log records from studies among five countries. Results show that response rate and completion status for interactive voice response (IVR) surveys vary between countries, survey types, and survey topics. The complexity of question trees, the number of question blocks in a tree, and the total call duration are relevant parameters to improve response and survey completion rate. One of the main advantages of IVR surveys is low cost and time efficiency. The total cost for operating 1,000 calls of 5 min each in five countries was 1,600 USD. To take full advantage of 5Q, questions and question-logic trees must follow the principle of keeping surveys smart and simple and aligned to the project's theory of change and research questions. Lessons learned from operating the IVR surveys in five countries show that the response rate improves through quality control of the phone contact database, using a larger pool of phone numbers to reach the desired target response rate, and using project communication channels to announce the IVR surveys. Among other things, the respondent's first impression is decisive. Thus, the introduction and the consent request largely determine the response and completion rate.