2026 Case Study

2026 Case Study

Introduction

In May-July 2026, we conducted 1,610 phone interviews with rice farmers in Pakistan. The average interview lasted 27 minutes. The broader survey examined working conditions and farmer well-being, including risks related to child labor, gender, and coercive labor practices.

  • Pakistan

    Location

  • May-Jul 2026

    Phone Survey

  • 1,610

    Rice farmers

  • 27 min

    Average Survey

The Survey Tool

Enumerator instruction: read every statement, then ask only for the total number that apply. Do not ask which specific statements are true. Depending on the list, the respondent answers with a number from 0-3 or 0-4.

Enumerator training before the real questions

Before asking the sensitive list questions, enumerators first explained the counting task and used neutral examples to make sure respondents understood the format.

“Thank you for your responses so far! The next questions are about typical farming practices on your rice farm. Now I have a few questions of a slightly different format for you - this is not the typical question & answer options format and I’ll explain it in a bit more detail. I will read a few statements, and your job is to simply tell me how many are true for you. This means you respond with a number between 0 and 3, or 0 and 4, depending on how many statements I read out.

Now before we get started, I would like to share an example of the types of questions I will ask you in this section. This one isn’t about rice farming, but I’m using it as a way to show you how these questions work.”

We also administered two practice questions to assess whether farmers understood the List Experiment methodology.

Worked Example

How the counting task was explained

  1. The sun rises in the morning.
  2. The sky is blue during the day.
  3. The sun shines at midnight.

Example explanation: the answer would be 2 because only the first two statements apply.

Practice Question 1

Test understanding (answer 0-4)

  1. You are a farmer.
  2. You grow rice.
  3. You live in China.
  4. You eat food and drink water.

The correct answer was three, as these farmers do not live in China.

Practice Question 2

Test understanding (answer 0-3)

  1. I did not sleep yesterday.
  2. I slept for less than or exactly 6 hours yesterday.
  3. I slept for more than 6 hours yesterday.

The correct answer was one, as the three options presented were mutually exclusive.

Listen to how the List Experiment works in action!

Enumerator

Thank you so much for your questions so far. They've been really helpful. We now move forward to list experiments. These are slightly different questions and where you will be answering me, not which statements are true. I'll read out you a set of statements. You will give me answers saying how many of them are applicable to you or your community. Okay, does that make sense? Should we get started?

Respondent

Okay.

Enumerator

So, given these are different questions, I'll get started with one example. I'll explain how these questions are asked and I'll give you the answer. So, for example, imagine I read you these three statements. Just listen. So, these are like one, the sun rises in the morning. Second statement, the sky is normally blue during the day. Third statement, the sun shines at midnight. So, the first statement is sun rises in the morning. Second statement is sky is normally blue during the day and third is sun shines at the midnight. So, when I read out these statements and ask you to give an answer, you tell me how many of these are applicable to you. You don't have to tell me which are correct. So, in this case, only two statements are correct because sun do not shine at the midnight. Right? So, only the first statement sun rises in the morning and sky is normally blue during the day. Only these two are correct. So, yeah. Were you able to follow me on the questions like how to answer?

Respondent

So, my answer should be two. Two.

Enumerator

You just have to tell me two. You do not have to validate which statements are correct. So, you should not be telling that first statement is correct, second statement is correct, third statement is wrong. You don't have to explain. You just give me the final number.

Respondent

Okay? Okay. Yeah, that's great.

Enumerator

Then we'll start with an actual one example question. So, I'll again read four statements. You can tell me the answer between zero to four. First statement, you're a farmer. Second statement, you grow coffee. Third statement, you live in Kenya. Fourth statement, you eat food and drink water. So, I'll repeat. First statement, you're a farmer. Second statement, you grow coffee. Third statement, you live in Kenya. Fourth statement, you eat food and drink water. Can you tell me like how many of these are applicable or correct for you?

Respondent

Four. Yeah.

Enumerator

Let me read again because see here. First statement says you're a farmer, which is correct. Second statement says you grow coffee. That's correct. Third statement says you live in Kenya, but you live in India. This should be wrong statement. In this case, only three are applicable for you. So, you should just tell me only three statements are correct.

Respondent

Okay. I understand. Yeah.

Enumerator

I hope you've got the question. So, yeah, you will just tell me how many of these are correct. I mean, I can repeat the statements if needed. Okay. Yeah.

Respondent

Sure.

Enumerator

Let's do one more example before the actual list questions. So, I'll read out again three statements. You give me an answer. How many of these are correct? You can tell me zero, one, two or three. That's it.

Respondent

Okay. Okay. I'll read the statements.

Enumerator

First statement, you did not sleep at all yesterday. Second statement, you slept for six hours or less yesterday. Third statement, you slept for more than six hours yesterday.

Respondent

Can you repeat again? Okay. I just need a second report. Let me repeat.

Enumerator

The first statement, you did not sleep at all yesterday. Second statement, you slept for six hours or less. Third statement, you slept for more than six hours.

Respondent

Okay. So, my answer would be one.

Enumerator

Exactly. Yeah. Because among those three statements, only one is correct, right? So, either you would have slept six hours or less or you would have slept six hours or more than seven hours. So, yeah, all the three statements or two statements are not possible. Only one is the right answer. So, you have given me the correct answer. So, this is our actual list questions. So, I'll read out the three or four statements and you tell me how many of these will be applicable to you.

Respondent

Okay. Okay. Okay.

Enumerator

Let's get started with the actual question. Again, I repeat, you don't have to tell me which of these statements are true. You just have to give me how many are correct. That's it.

Respondent

Okay. Understood. Yeah.

Enumerator

So, in this question and few more, we will be asking you that we will have statements about you or your households or your community or your coffee farm. Some may be applicable to you or some may not. And please listen to all of these statements and tell me how many of these are applicable to you.

Respondent

Okay. Okay. Okay. Let me start.

Enumerator

So, I'll read out four statements. You can maybe use your fingers to count these statements. One, women in your community typically take care of the household chores. Two, men in your community are usually seen as the head of the household. Three, women in your community face behavior of violent or unwanted sexual in nature when working on the coffee farms. Fourth, women and men in your community spend the same amount of time working on the farms. I'll repeat. So, you have to tell me from these four statements, how many are correct? Zero, one, two or three. I'll repeat the statements. One, women in your community typically take care of the household chores. Two, men in your community are usually seen as head of the household. Three, women in your community face behavior of violent or unwanted sexual nature when working on the coffee farms. Four, women and men in your community spend the same amount of time working on the farms.

Respondent

Yeah.

Enumerator

Let me know how many of these are applicable. Don't have to tell me which are the correct and which are the wrongs. Just give me between zero to four how many of these are correct.

Respondent

So, three is my answer. Okay. Thank you so much.

Enumerator

We can move forward to the next question.

Respondent

Okay.

Findings

Gendered Wage Discrimination

Please share how many of the following statements apply to your community.

Do not specify which ones, just tell me how many total statements apply.

Control List
  1. 1NeutralIt is common for family members to help on the farm.
  2. 2NeutralIt is common for men and women to perform similar types of tasks on the farm.
  3. 3DisagreeableIt is common for farm laborers to decide their pay.
  • 0
  • 1
  • 2
  • 3
Treatment List
  1. 1NeutralIt is common for family members to help on the farm.
  2. 2NeutralIt is common for men and women to perform similar types of tasks on the farm.
  3. 3NeutralIt is common for farm laborers to decide their pay.
  4. 4SensitiveIt is common for women to be paid differently than men for the same type of work.
  • 0
  • 1
  • 2
  • 3
  • 4

Direct Verification Question

Do women working on rice farms get paid differently than men for the same type of work?

Question sample: n = 1,527

Results

  • 16%

    Direct Question

  • 24%

    Indirect List Estimate

  • 2x

    Factor of Difference

What We Learned

1 in 4 farmers indirectly report that men and women are commonly paid different amounts for the same work. Indirect reporting was higher than direct reporting (24% vs. 16%).

Gendered Violence

Please share how many of the following statements apply to your community.

Do not specify which ones, just tell me how many total statements apply.

Control List
  1. 1NeutralWomen in your community typically take care of household chores.
  2. 2NeutralWomen in your community travel freely when they want to without informing household members.
  3. 3DisagreeableWomen and men in your community spend the same amount of time working on the farms.
  • 0
  • 1
  • 2
  • 3
Treatment List
  1. 1NeutralWomen in your community typically take care of household chores.
  2. 2NeutralWomen in your community travel freely when they want to without informing household members.
  3. 3SensitiveWomen in your community face unwanted physical behavior or inappropriate language while working on rice farms.
  4. 4DisagreeableWomen and men in your community spend the same amount of time working on the farms.
  • 0
  • 1
  • 2
  • 3
  • 4

Notes to researchers: Household chores include cleaning, cooking, and washing clothes. Unwanted behavior refers to unwanted touching, inappropriate comments, threats, hitting, or physical or verbal abuse that makes a person feel unsafe or uncomfortable.

Direct Verification Question

Do women working on rice farms in your community face unwanted physical behavior or inappropriate language?

Question sample: n = 1,527

Results

  • 3%

    Direct Question

  • 12%

    Indirect List Estimate

  • 4x

    Factor of Difference

What We Learned

Around 12% indirectly say that women in their community experience unwanted physical contact or inappropriate language.

Child Labor

Please share how many of the following statements apply to you, your household and your rice farm.

Do not specify which ones, just tell me how many total statements apply.

During the last rice season, to do some work on my farm (consider all types of activities):

Control List
  1. 1NeutralAt least one other adult in my family helped me on the farm.
  2. 2NeutralI have not employed any adult woman over 18 on the farm.
  3. 3DisagreeableI employed a group of community members from my village who work on the farm.
  • 0
  • 1
  • 2
  • 3
Treatment List
  1. 1NeutralAt least one other adult in my family helped me on the farm.
  2. 2NeutralI have not employed any adult woman over 18 on the farm.
  3. 3SensitiveI employed someone under the age of 15 who is not my family member to help me.
  4. 4DisagreeableI employed a group of community members from my village who work on the farm.
  • 0
  • 1
  • 2
  • 3
  • 4

Direct Verification Question

During the last rice planting season, to do some work on your farm (consider all types of activities), did at least one person under the age of 15 who is not your family member help you?

Question sample: n = 1,527

Results

  • 14%

    Direct Question

  • 20%

    Indirect List Estimate

  • 1.5x

    Factor of Difference

What We Learned

The indirect estimate was higher than direct reporting (20% vs. 14%). One in five farmers were estimated indirectly to have had a non-family worker under 15 help on the farm.

Child School Attendance

Please share how many of the following statements apply to you or your household regarding children below the age of 15.

Do not specify which ones, just tell me how many total statements apply.

Control List
  1. 1NeutralChildren play with other children in the community in the evenings.
  2. 2NeutralChildren decide on their own if they want to attend school or not.
  3. 3DisagreeableChildren help with household chores after school to learn important life skills.
  • 0
  • 1
  • 2
  • 3
Treatment List
  1. 1NeutralChildren play with other children in the community in the evenings.
  2. 2NeutralChildren decide on their own if they want to attend school or not.
  3. 3NeutralChildren help with household chores after school to learn important life skills.
  4. 4SensitiveChildren help out on the farm rather than go to school when there is important farm work to be done.
  • 0
  • 1
  • 2
  • 3
  • 4

Direct Verification Question

Do children help out on the farm rather than go to school when there is important farm work to be done?

Question sample: n = 1,527

Results

  • 10%

    Direct Question

  • 7%

    Indirect List Estimate

  • No detectable difference

    Factor of Difference

What We Learned

1 in 10 farmers believe children should work on the farm instead of attending school when important work needs to be done. There was no detectable difference between indirect and direct questioning (7% vs. 10%). In this study, indirect questioning did not reveal higher reporting for this issue.

Children Mixing Agro-chemicals

Please share how many of the following statements apply to you or your rice farm.

Do not specify which ones, just tell me how many total statements apply.

During the last rice season, to do some work on my farm (consider all types of activities):

Control List
  1. 1NeutralAt least one family member over the age of 18 helped either mix or spray agrochemicals.
  2. 2NeutralNo male mixed or sprayed agrochemicals.
  3. 3DisagreeableHired laborers helped mix or spray agrochemicals.
  • 0
  • 1
  • 2
  • 3
Treatment List
  1. 1NeutralAt least one family member over the age of 18 helped either mix or spray agrochemicals.
  2. 2SensitiveChildren under the age of 18 mixed or spray agrochemicals.
  3. 3NeutralNo male mixed or sprayed agrochemicals.
  4. 4DisagreeableHired laborers helped mix or spray agrochemicals.
  • 0
  • 1
  • 2
  • 3
  • 4

Note: Agrochemicals refers to chemical pesticides, herbicides, insecticides, etc.

Direct Verification Question

Did children under the age of 18 help mix or spray agrochemicals on your rice farm?

Question sample: n = 1,527

Results

  • 3%

    Direct Question

  • 0%

    Indirect List Estimate

  • No detectable difference

    Factor of Difference

What We Learned

It is uncommon for children under 18 help mix or spray agrochemicals. Reporting was very low and there was no detectable difference between indirect and direct questioning (0% vs. 3%). The study noted that a much larger sample may be needed to detect differences for such a risk.

Threats

Please share how many of the following statements apply to your community.

Do not specify which ones, just tell me how many total statements apply.

Control List
  1. 1NeutralWorkers in my community must finish their work within a fixed amount of time on the farm.
  2. 2NeutralWorkers in my community are closely watched by their employers when working on the farm.
  3. 3DisagreeableWorkers in my community are free to decide when to stop working on the farm.
  • 0
  • 1
  • 2
  • 3
Treatment List
  1. 1NeutralWorkers in my community must finish their work within a fixed amount of time on the farm.
  2. 2NeutralWorkers in my community are closely watched by their employers when working on the farm.
  3. 3NeutralWorkers in my community are free to decide when to stop working on the farm.
  4. 4SensitiveWorkers in my community are treated aggressively or threatened if they do not finish their work on the farm.
  • 0
  • 1
  • 2
  • 3
  • 4

Illustrative survey visual for List Experiment administration.

Direct Verification Question

Are workers in your community treated aggressively or threatened if they do not finish their work on the farm?

Question sample: n = 1,054

Item added after the first 556 interviews.

Results

  • 14%

    Direct Question

  • 28%

    Indirect List Estimate

  • 2x

    Factor of Difference

What We Learned

28% of farmers indirectly report that workers in their community face aggression if they do not complete their farm work. The indirect estimate was twice the direct result (28% vs. 14%), revealing a substantial reporting gap for threats or aggression.

2026 Case Study

Action items

Awareness

Strengthen awareness among farmers, workers, and local partners of the identified risks, including child labor, gender-based discrimination and violence, restrictions on workers’ freedom, and hazardous working conditions. Training should go beyond general awareness-raising to provide practical examples of unacceptable practices, clarify expected standards of conduct, and highlight the channels available to farmers and workers to raise concerns. ILO guidance (opens in a new tab) similarly emphasizes awareness, practical tools, and capacity-building as part of identifying and preventing labor-rights risks.

Policy Interventions

Translate the findings into clear policies and expectations for farms and suppliers, covering minimum-age requirements, restrictions on hazardous work, equal pay, freedom to leave employment, protection against threats and retaliation, and accessible grievance and remediation mechanisms. The OECD-FAO Guidance (opens in a new tab) recommends integrating responsible business conduct standards into organizational policies and adopting risk-based due diligence to identify, prevent, mitigate, and address adverse impacts across agricultural supply chains.

Incentives and Certification

Leverage certification, verification, and commercial incentives to promote safer and fairer practices. This could include recognizing farms that demonstrate improvements or linking additional support to compliance with labor standards. However, certification should complement rather than replace due diligence. Its effectiveness depends on aligning certification standards, verification processes, supplier support, and remediation measures with identified risks. The OECD-FAO Guidance also recommends assessing whether third-party certification schemes align with risk-based due diligence expectations.

Ongoing Monitoring

Repeat the assessment every 3–5 years, using consistent questions, comparable sampling approaches, and similar agricultural seasons to track changes in risks and reporting behavior over time. More frequent monitoring may be warranted where particularly serious risks are identified. While the proposed 3–5-year interval is specific to this assessment rather than an internationally prescribed standard, external guidance emphasizes the importance of ongoing monitoring and verification to assess whether interventions are effectively reducing risks.

2026 Case Study

Limitations

List Experiments give respondents a safer way to report sensitive experiences and can reveal risks that direct questions miss. But like any method, they have limitations, from study design to how results are interpreted.

Understanding and documenting these limitations helps readers judge how much weight an estimate can bear, avoid over-interpreting small or absent differences, and identify where other evidence is needed. Each limitation below includes practical steps to reduce its impact.

List Experiments place higher cognitive demands on respondents

List Experiments require respondents to listen to or read several statements, privately determine which ones apply to them, and report only the total number. This is more demanding than answering a straightforward yes-or-no question, particularly when statements are read aloud and respondents have limited formal education. Long surveys can further increase respondent fatigue, making it harder to follow the counting exercise accurately.

What can we do? Use simple, clearly worded statements and provide a practical example before introducing the List Experiment. Include comprehension questions to check whether respondents understand the counting exercise, and allow sufficient time for them to consider each statement. Monitor comprehension rates throughout data collection and provide additional enumerator guidance or retraining where respondents consistently struggle.

The inclusion of a sensitive statement can influence how respondents perceive other statements

List Experiments assume that adding a sensitive statement does not change how respondents answer the non-sensitive statements. However, this assumption may not always hold. The presence of a sensitive statement may make respondents more cautious or defensive, or lead them to interpret the other statements differently. This is known as a design effect.

When a design effect occurs, the difference between treatment and control group averages may reflect both the prevalence of the sensitive behavior and changes in how respondents answer the other statements, resulting in a biased estimate. Although statistical tests can help identify design effects, they may not always detect them reliably (Blair and Imai, 2012).

What can we do? Carefully design non-sensitive statements so they are neutral, easy to understand, and related to the broader topic without drawing unnecessary attention to the sensitive behavior. Pilot the full list with the target population to identify whether the sensitive statement changes how respondents interpret or answer the other items. During analysis, check for potential design effects and interpret findings cautiously, recognizing that the absence of a statistically significant design effect does not necessarily mean that none exists.

List Experiments estimate group-level prevalence rather than individual behavior

List Experiments are designed to estimate the proportion of respondents who have experienced or engaged in a sensitive behavior, without identifying which individuals it applies to. While this provides respondents with greater privacy, it also limits the level of analysis that can be conducted.

In particular, researchers cannot directly link the sensitive behavior to individual respondent characteristics in the same way they could with a direct question. Understanding differences across subgroups, such as women farmers, particular geographic areas, or different education levels, typically requires larger samples and more advanced statistical methods.

What can we do? Clearly define the population and key subgroups of interest before designing the study, and ensure the sampling strategy provides sufficient representation of these groups. Where subgroup comparisons are important, account for the additional sample size and analytical requirements during project planning. Interpret findings primarily at the population level and avoid drawing conclusions about individual respondents or subgroup differences when sample sizes are insufficient.

It is difficult to know in advance what farmers will perceive as sensitive

Researchers may identify a topic as sensitive based on the nature of the issue, but farmers may not perceive it in the same way. Equally, a topic that appears straightforward to researchers may carry more stigma or perceived risk in a particular local context. Sensitivity is shaped by social norms, cultural expectations, and what respondents believe could happen if they disclose certain information.

This creates an important limitation for List Experiments. If the method is treated as a one-stop solution and implemented at full scale based only on assumptions about sensitivity, researchers may complete the entire exercise only to find little or no difference between direct and indirect reporting. In that case, the List Experiment has added considerable survey complexity, respondent time, sample requirements, and cost without generating additional insight.

What can we do? Treat List Experiments as an iterative process rather than a fixed survey module. Direct verification is particularly important because it allows researchers to compare indirect and direct reporting as data collection progresses. If early results consistently show little or no difference between the two approaches, this can signal that the topic may not be particularly sensitive in that population and allow for course correction before further resources are committed.

Piloting and early-stage analysis should therefore be used to test both respondent understanding and whether the indirect approach is adding meaningful value. Questions can then be refined, replaced, or removed where appropriate before scaling the List Experiment further.

List Experiments are resource-intensive

List Experiments require considerable time, quality control, budget, and high sample size.
They often require large samples to detect differences between treatment and control groups, alongside highly trained researchers who can administer the format consistently and test respondent comprehension. Analysis is also more complex, requiring additional statistical calculations and specialist knowledge to correctly estimate and interpret the results.

In practice, this means several team members may need to spend substantial time on researcher training, fieldwork monitoring, debriefs, quality checks, data review, and analysis over an extended period.

What can we do? Allocate more time to each stage of the project than you would for a standard survey. Use experienced researchers who are already comfortable with complex survey administration, create clear checklists and protocols to streamline training and quality checks, and build sufficient time for monitoring, analysis, and troubleshooting into the project plan. Adequate time and skilled personnel are essential to implementing the method well.

2026 Case Study

Key Learnings

Our experience implementing List Experiments across agricultural supply chains has highlighted several practical considerations that can influence the effectiveness of this methodology. The following learnings reflect some of the key considerations and best practices identified through implementation.

  1. 01

    Designing and testing non sensitive statements is as important as sensitive statements

    The selection of non-sensitive statements can significantly influence the reliability of List Experiment results. Statements that are unfamiliar, ambiguous, or unrelated to the broader sensitive theme may confuse respondents or make the sensitive statement more noticeable.

    Another important consideration is avoiding ceiling and floor effects, where respondents consistently select all or none of the statements. For example, if a large proportion of respondents select all three statements in the control group or all four in the treatment group, this may indicate that the non-sensitive statements are too universally applicable. In particular, it may suggest that the deliberately less-common statement is not functioning as intended. Such patterns can compromise the confidentiality provided by the List Experiment and reduce the reliability of estimates.

    What can we do? Develop simple, culturally relevant statements that reflect respondents’ everyday experiences and relate to the broader topic being investigated. Include a mix of commonly applicable and less-common but plausible statements to encourage variation in responses. During piloting and data collection, monitor the proportion of respondents selecting all or none of the statements. A high concentration of these responses should prompt a review of the question framing and the suitability of the non-sensitive items.

  2. 02

    Treatment and control group patterns can reveal design inaccuracies during data collection

    List Experiment estimates depend on comparing the average number of statements selected by the treatment and control groups. Unexpected differences between these groups may indicate problems with question design, respondent comprehension, or survey administration.

    For example, a control-group average that exceeds the treatment-group average is counterintuitive under the standard List Experiment assumptions and warrants further investigation. Such patterns may also be concentrated among particular respondent groups rather than appearing consistently across the full sample.

    What can we do? Periodically calculate and compare treatment and control group means while data collection is underway. Examine results by relevant respondent characteristics, such as geography, gender, education level, or farmer profile, to identify whether unusual patterns are concentrated in specific groups. These checks should be conducted early enough to investigate and address potential issues before fieldwork concludes.

  3. 03

    Investing in experienced enumerators, thorough training, and consistent administration upfront is valuable

    The effectiveness of List Experiments depends heavily on how questions are administered. Given the complexity of the methodology, it is important to engage enumerators who have prior experience administering complex surveys and ensure they are well-versed in the List Experiment approach. Differences in wording, tone, translation, or explanations provided by enumerators can influence respondent understanding and introduce measurement errors.

    Unusual response patterns concentrated among particular enumerators may indicate inconsistent administration, recurring points of confusion, or contextual factors affecting how respondents interpret the questions.

    What can we do? Prioritize recruiting experienced enumerators and invest in comprehensive training, including practice interviews, role-playing exercises, and practical techniques for explaining the methodology and addressing respondent confusion without influencing their answers. During data collection, monitor enumerator-level response patterns and investigate anomalies with field teams or supervisors. Where interviews are recorded with appropriate consent, spot-check a sample to assess whether enumerators follow the script, allow sufficient thinking time, and administer questions consistently.

    Use these findings to provide targeted feedback and refresher training, and consider replacing enumerators where persistent inconsistencies or enumerator effects are identified and cannot be resolved through additional support.