# Avoiding Bias in Data Collection 🧐 Bias occurs when certain responses are systematically favored over others. 👎 Bias occurs when certain responses are systematically favored over others. 👎 Bias occurs when certain responses are systematically favored over others. 👎 Bias occurs when certain responses are systematically favored over others. 👎 **Bias occurs when certain responses are systematically favored over others**, either intentionally or unintentionally. That's right! Previously, we've discussed methods of sampling accurately but now, it is imperative to discuss how *not* to gather data. Statistical studies are **biased** if it is likely to underestimate or overestimate the value you are looking for. Bias can result in a sample or a study that is not representative of the population being studied, which can lead to inaccurate or misleading conclusions. 😕 ## Bias in Sampling ### Voluntary Response **Voluntary response bias**, also known as **self-selection bias**, occurs when a sample is comprised entirely of volunteers or people who choose to participate in a study. This can occur when the study relies on self-reported data or when participants are recruited through means such as online surveys or television or radio programs. 🙋‍♂️ Because voluntary response samples are not randomly selected, they are often not representative of the population being studied. For example, people who are more interested in the topic being studied or who have strong opinions on the topic may be more likely to volunteer to participate in the study. This can result in a sample that is biased and does not accurately represent the views or characteristics of the population. #### Example You are a researcher who is interested in studying the attitudes of young adults towards social media. You decide to conduct an online survey to gather data on this topic. You create a link to the survey and post it on your personal social media accounts, as well as on several online forums and discussion groups related to social media. You also share the link with your friends and ask them to share it with their friends. Within a few days, you receive a large number of responses to your survey. However, as you begin to analyze the data, you realize that there are some problems with the sample. Many of the respondents are young adults who are very active on social media and have strong opinions about it, while others are older adults who do not use social media as much. There are also a disproportionate number of responses from people who live in urban areas, while responses from people in rural areas are much less common. Upon further investigation, you realize that the sample for your survey is not representative of the population of young adults in the United States. Instead, it is comprised mostly of volunteers who chose to participate in the survey, and these volunteers are not representative of the overall population. 😔 ### Undercoverage **Undercoverage bias** occurs when a portion of the population has a reduced chance of being included in the sample, resulting in a sample that is not representative of the population. This can occur for a variety of reasons, such as when the sampling frame is incomplete or when certain groups are more difficult to reach or sample than others. 😡 #### Example if you're studying the attitudes of college students towards climate change and you use a sampling frame that only includes students at four-year colleges and universities, you may be introducing undercoverage bias into your sample. This is because students at two-year colleges and trade schools may be underrepresented in your sample, even though they are part of the overall population of college students in the United States. 😔 ### Nonresponse **Nonresponse bias** occurs when individuals who are chosen for the sample but for whom data cannot be obtained (or who refuse to respond) differ from those for whom data can be obtained. This can happen when a study relies on self-reported data or when data is collected through methods such as surveys or interviews. 👻 #### Example If you're conducting a survey of college students' attitudes towards climate change and you send the survey to a random sample of 1000 students, but only 500 students complete and return the survey, you may be introducing nonresponse bias into your sample. This is because the students who completed and returned the survey may differ in some way from the students who did not respond, even though both groups were chosen for the sample. ### Wording in Questions **Question wording bias** occurs when the wording of a survey question or the way a question is asked influences the response that is given. This can happen when the question is confusing, misleading, or ambiguous, or when the question asks for too much or too little information. ✉️ #### Example Consider the following two questions: - "Do you think that the government should provide free healthcare for all citizens?" - "Would you support a government healthcare program that would provide free healthcare for all citizens?" These two questions are asking about the same topic, but they are worded differently. The first question uses a leading phrase ("Do you think that") that may bias the respondent towards agreeing with the statement, while the second question presents the idea of a government healthcare program more neutrally. As a result, the responses to these two questions may differ, even though they are asking about the same topic. ### Convenience Sampling **Convenience sampling** occurs in which the sample is selected based on *ease of access* or *availability*, rather than through a random selection process. This means that the sample is not representative of the population and may be biased. 🥪 #### Example Let's go back to the researcher who wants to study the attitudes of college students towards climate change! Rather than using a random sampling method to select a representative sample of students, the researcher decides to conduct the study on a college campus that is close to their home. The researcher then interviews students who are available and willing to participate, such as students who are passing by or who are hanging out in a common area. In this example, the sample is not representative of all college students in the United States, but rather a convenience sample of students from a single college campus. This sample may be biased because it only includes students who are available and willing to participate, and it does not include students who are not on campus or who are not interested in participating in the study. As a result, the results of the study may not accurately represent the attitudes of college students towards climate change. ## Why does this matter, then? If sampling is done without considering the factors that could invalidate your results, then the concluding data and analysis will not be representative of the population. ![Image of a tweet about sampling bias](https://i.ibb.co/0fLRPmm/Ci-Ot-O9f-VEAAB3-RU.jpg) ###### Courtesy of Twitter **Always** consider how the sampling method could introduce bias into your study. Think about who might be excluded or overrepresented in the sample. Identifying potential biases is a key skill for the AP exam. ## Practice Problem You are a researcher who is interested in studying the attitudes of small business owners towards the economy. You decide to conduct a survey to gather data on this topic. To collect data for the survey, you create a list of all the small business owners in your city and send the survey to a random sample of 1000 business owners. However, you only receive responses from 500 of the business owners, and many of the respondents are from larger businesses with more than 50 employees. Upon further analysis, you realize that your sample is not representative of the population of small business owners in your city. **a) Identify the type of bias present in this sample.** **b) Explain how this bias may have affected the results of the survey.** ### Answer **a)** In this situation, the sample is suffering from **nonresponse bias** because not all of the business owners who were chosen for the sample responded to the survey. This means that the sample is not representative of the population of small business owners in the city, as it only includes responses from 500 business owners, many of whom are from larger businesses. **b)** This bias may have affected the results of the survey in several ways. For example, the attitudes of small business owners who did not respond to the survey may differ from those who did respond, even though both groups were chosen for the sample. Additionally, the *oversampling* of larger businesses may have skewed the results in a way that is not representative of the attitudes of small business owners overall. As a result, the results of the survey may not accurately represent the attitudes of small business owners towards the economy. **Multiple Choice Questions** 1. A researcher wants to study the exercise habits of adults in a city. They decide to survey people at a local gym. What type of bias is most likely to be present in this study? (A) Voluntary response bias (B) Undercoverage bias (C) Nonresponse bias (D) Convenience sampling bias (E) Wording bias 2. A survey question asks, "Given the overwhelming evidence, do you agree that climate change is a serious problem?" What type of bias is most likely to be present in this survey? (A) Voluntary response bias (B) Undercoverage bias (C) Nonresponse bias (D) Convenience sampling bias (E) Wording bias **Free Response Question** A school is conducting a survey to understand student opinions on a new lunch menu. They randomly select 200 students from the school roster. Of those selected, 120 students return the survey. The survey includes a question that asks: "Do you think the new lunch menu is an improvement over the old one?" (a) Identify a type of bias that could be present in this scenario. (b) Explain how this bias might affect the results of the survey. (c) Suggest one way the school could improve its survey method to reduce bias. **Scoring Guidelines** **(a) (1 point)** * Correctly identifies nonresponse bias or wording bias. **(b) (2 points)** * Explains how nonresponse bias could occur if the students who responded have different opinions than those who did not. * Explains how the wording of the question could lead students to answer in a particular way. **(c) (1 point)** * Suggests a valid method to reduce bias, such as using a more neutral question or following up with non-respondents. **Example Answers** **(a)** Nonresponse bias or wording bias are both acceptable. **(b)** Nonresponse bias could occur because the 80 students who did not respond to the survey may have different opinions about the lunch menu than the 120 students who did respond. For example, students who dislike the new menu may be less likely to respond. The wording of the question may also lead to bias because it implies that the new menu should be an improvement, which could cause students to respond more favorably. **(c)** To reduce bias, the school could reword the question to be more neutral (e.g., "What do you think of the new lunch menu?") or follow up with the students who did not respond to the first survey to try to get their opinions.

Potential Problems with Sampling

Ava Garcia

How are we doing?

Potential Problems with Sampling

Ava Garcia

How are we doing?