Residents may voice their opinions on how city officials are using the skewed results of a poorly-designed and biased survey to plan the future of their city by contacting their city council members.

PHENIX CITY, Ala. — City officials are using an inherently flawed public survey to guide development decisions for the city’s future.

The survey and its data collection methods contain serious design flaws, making it only capable of producing heavily skewed and largely misleading results. City officials will use those inherently skewed results to make decisions on how to plan the city’s future.

The “Visioning Survey” is intended to help the city guide its development of its new “Fast Forward Phenix City” project: a self-proclaimed “updated city-wide plan” for comprehensive future development efforts. 

The survey is being distributed through the city’s “Positively Phenix City” Facebook page. For users to even be served the opportunity to participate in the survey, they must by default already be a follower of that page. 

SAMPLE BIAS

This distribution method presents an instance of what is known as sample bias, where a specific group of people are systematically more likely to be chosen to complete the survey than others. When a survey receives more responses from a specific group of people, the results are skewed to that group’s responses. 

For example: if you wanted to know what sort of weather is preferred by animals, but you only asked the ducks, your survey results would falsely show that all animals prefer rainy days; you applied a sample bias by only asking ducks and applied their duckly responses to all animals. If you want to know what weather animals prefer, you have to ask all types of animals equally. 

In the case of the city’s Visioning Survey, the survey is systematically far more likely to be completed by those who already follow Positively Phenix City’s Facebook page. Following our example above, the “ducks” in this case are those who follow the Facebook page; they’re the only people to whom the survey questions are being asked.

Because of this sample bias being built-in to the survey’s design, the survey’s results will — by-and-large — be immensely skewed to reflect the voice of those who already follow Positively Phenix City’s Facebook page. 

Given that those who follow the Facebook page are — by-and-large —  those who opted-in to support the city’s development efforts by following the page, the survey’s results will be systematically skewed to reflect a positive light on the city’s past efforts; the survey is only shown to people who already opted-in their support, and it then asks them about things they are already known to feel favorable about.

Since the city’s Visioning Survey asks participants about how favorably they view the city’s past development efforts, and since those results will be systematically skewed to reflect the favorable views of those who are already known to support those same efforts, city officials will — by default — be misguided to believe the public thinks more highly of their performance than it actually does; they have only asked the ducks, and have used those duckly responses to believe that all animals prefer the rain. 

RESPONSE BIAS

The positive-sentiment mentality of the group of people systematically chosen to complete the survey through its inherent sample bias also presents another problem: response bias.

Response bias is when a survey’s participants are more likely to provide answers that differ from how they actually feel. While there are seven different types of response biases, the one we are most concerned about in case of the city’s Visioning Survey is known as the social desirability bias.

Social desirability bias occurs when survey participants are more likely to provide responses they believe will be acceptable by their social group, regardless of how they themselves actually feel as an individual.

Let’s view the social desirability bias through our “duck” metaphor. For example: A flock of ducks is deciding where to fly to next based on what weather is preferred by most ducks. To find out, the “lead duck” asks what type of weather the ducks prefer to fly through. The “popular” ducks who fly near the leader immediately respond by saying they love the rain. However, most of the ducks have seen hunters with shotguns on rainy days and know it’s best to avoid flying through the rain at all costs. Since those ducks know they depend on the group for their survival, the “no-rain” majority falls victim to the social pressure of the “popular ducks” and all say they prefer rainy days anyway. With skewed results showing that every duck preferred rainy days, the flock flew through the rain and were promptly decimated by the hunters waiting below.

In the case of the city’s Visioning Survey, and given the sample bias we discussed earlier, we know that those who follow the Positively Phenix City Facebook page are inherently more likely to set the social tone for how others in their “popular duck” social groups view the city’s past performance. 

Since the “popular ducks” have been systematically selected through the survey’s sample bias as described above, they are therefore the only ones capable of encouraging others in their social circles to take the survey as well.

In doing so, they spread a preconceived notion of “positivity” through Phenix City residents, whether those residents actually feel positively about the city’s efforts or not. When those residents take the survey, they are inherently more likely to provide results that align with the dominant views of the “popular ducks” who set the tone for that social group. In the case of the Vision Survey, this means that social pressures are highly likely to produce results that are heavily skewed toward favoring the city’s past performance, whether survey participants actually believe their own responses or not. 

NON-RESPONSE BIAS

At the time of this article’s writing, only 34 individual people have responded to the city’s Visioning Survey. Given that the Positively Phenix City Facebook page has a following of 3,800 people, this means that just under 1% of them have actually chosen to complete the survey. 

This small response presents what is known as a non-response bias, which is when those systematically selected to participate in a survey choose not to respond. 

In short: Even within the group of “popular ducks” who were systematically selected to be shown the survey, only one out of every hundred of them even bothered to take the survey. This seems self-explanatory enough; no “duck metaphor” required.

Nonetheless, the responses provided by 34 individual people — less than 0.1% of the population; less than one-in-one-thousand — will be used by city officials to guide their planning for the city’s future.

SUMMARY

The biases within the city’s Visioning Survey are systematically designed to produce results that favor a more positive outlook of the city’s past performance. Those skewed results will then be used by city officials to plan the future of the city. That misguided planning will then produce more of the same, while city officials cite “data” as the reason for their decision-making — and the cycle of ignorance continues, through a system that is only ever capable of producing the same results it previously has. 

Congratulations, Phenix City officials. You’ve incompetently created a system that will mathematically always tell you that you’re doing a great job, whether you actually are or not, all while being able to blame “data” for your “justified” decisions, even if the city is metaphorically burning to ash around you.

Perhaps city officials should look into that. 

Residents may voice their opinions on how city officials are using the skewed results of a poorly-designed and biased survey to plan the future of their city by contacting their city council members.

Facts are stubborn things — and we’ll keep publishing them, whether city officials like them or not.

-30-

© 2023 Muscogee Muckraker. All rights reserved.

‍