(Note: I’m struggling with creating an interesting title for this. Any thoughts?)
There are topics about which people will lie when questioned. We know that.
These topics have changed over the years. Gay sex used to fall into this category; in some regions, it still does. So do church attendance, domestic violence, incest, voting, monitoring what children do online, kinky sex, healthy eating, drug use and regular exercise.
What these diverse topics share is a clear social norm. People believe they are supposed to behave in a certain way, and just don’t want to own up to doing something different. The motive could be embarrassment, paranoia or any of a number of other choices. The bottom line is that when you ask people about these behaviors, you’re going to get some misinformation.
We know that, and its no excuse for avoiding the questions or ignoring the results. We simply know that the raw data has to be put into a statistical model before it can be reported.
Live Science provided an example of this today. They reported on a study by a grad student at the University of Nevada showing that people who are more religious are more likely to oppose legalization of marijuana than those who are less religious. Now, that wouldn’t be an earth-shaking finding if it were true. The problem is that the data show a relationship between church attendance and opposition but no relationship between “religiosity” and opposition. Its a tough double-negative, but people who feel they are more religious don’t oppose legalization more than others.
OK, so how could simple church attendance matter, but religiosity not matter? You would expect people who feel they are more religious to attend church more often, right? That’s not true in every case, certainly, but more often than not, that’s what you would expect.
The other possibility is that people just don’t report their behavior accurately. People who feel most strongly bound by social norms are more likely to misreport attending church and more likely say they oppose legalization.
Why does that matter? It means that as perceptions of what’s socially acceptable change, their attitudes toward church and toward legalization are likely to change quite rapidly. These aren’t strongly held beliefs, rather, they are simply mirroring what they think others around them feel.
The misrepresentation issue is one of the items that has been playing havoc with election polls this year. Even if a person reports accurately his/her preferred candidate, will they show up to vote? People know they are supposed to vote, so a certain number will say they definitely plan to vote when they know they won’t. It always happens.
For that reason, pollsters use “likely voting” models to estimate who will really vote. These models are usually based on past behavior (e.g., did you vote in 2012? in 2008?). That works when you don’t have major changes in turnout. This year, we are likely to have a lot of new Hispanic voters, and the models won’t work for them.
It also doesn’t help that most election surveys are conducted in English. Hispanic households blow them off regularly, and its a big question mark whether those who respond really represent the community as a whole.
Statistical weighting or a likely voter model won’t overcome bias in the selection of respondents for a survey. In fact, if bias is severe, it’s not even proper to calculation a +/- error margin for survey results.
Frankly, if the Hispanic community votes in Texas the way the Black community did in 2008, Texas would have a Democratic representation in Congress.
Personally, I don’t think anyone will have to stay up late on election night this year. But we’ll see. I could use a good night’s sleep.
Sources:
- Blaszczak-Boxe, Agata, “Are Religious People Less Likely to Support Marijuana Legalization?” LiveScience, 29 Deptember 2016. http://www.livescience.com/56320-support-for-legalizing-marijuana-linked-with-religion.html?utm_source=ls-newsletter&utm_medium=email&utm_campaign=20160930-ls
- Keeter, Scott, and Igielnik, Ruth, “Can Likely Voter Models Be Improved?” Pew Research Center, 7 January 2016. http://www.pewresearch.org/2016/01/07/can-likely-voter-models-be-improved/
CRAIN'S COMMENTS 