|
Source: (consider it)
|
Thread: Polling data based on lies
|
deano
princess
# 12063
|
Posted
Following on from the recent threads on the Scottish referendum and the fall out from it, here is an interesting article on the BBC's Magazine site...
Small Data: Do people lie in surveys?
It reinforces my view that polls are pretty worthless in extracting the truth. Nobody wants to admit voting for someone who is perceived as unpopular, but when in the voting booth, with nobody to see them, they put their cross against the unpopular option.
There are few people who admit to voting for Mrs Thatcher in the 80's (I did), but she won three terms in office so somebody (other than me) must have done.
Again, nobody wanted to admit to voting for John Major in the early 90's, but he won.
The polls for the Scottish referendum showed the result should have been much closer than it was.
Would shipmates lie about this on polls? Given the left is perceived as "nice", and therefore popular, I suspect most shippies will say they don't lie, and they do actually vote the left-wing ticket.
But does the above article shake your confidence that the electorate is apt to be swayed towards actually voting left, as opposed to saying they will to appear "nice" but not doing so in the privacy of the voting booth?
The sort of thing highlighted in the article is why we on the right don't take much notice of polls. Better to have the ability that Mrs Thatcher had of being able to "sniff the wind" and instinctively know what the middle-classes want.
--------------------
"The moral high ground is slowly being bombed to oblivion. " - Supermatelot
Posts: 2118 | From: Chesterfield | Registered: Nov 2006
| IP: Logged
|
|
LeRoc
Famous Dutch pirate
# 3216
|
Posted
quote:
deano: Following on from the recent threads on the Scottish referendum and the fall out from it, here is an interesting article on the BBC's Magazine site...
IIRC, the polls predicted a small 'No' victory. What we got was ... a small 'No' victory. I'd say they were pretty much spot on.
quote:
deano: It reinforces my view that polls are pretty worthless in extracting the truth. Nobody wants to admit voting for someone who is perceived as unpopular, but when in the voting booth, with nobody to see them, they put their cross against the unpopular option.
Good polls normally correct for this.
quote:
deano: There are few people who admit to voting for Mrs Thatcher in the 80's (I did), but she won three terms in office so somebody (other than me) must have done.
I'm sure that polls in the eighties predicted a Thatcher victory. Which is exactly what happened.
--------------------
I know why God made the rhinoceros, it's because He couldn't see the rhinoceros, so He made the rhinoceros to be able to see it. (Clarice Lispector)
Posts: 9474 | From: Brazil / Africa | Registered: Aug 2002
| IP: Logged
|
|
Sioni Sais
Shipmate
# 5713
|
Posted
I don't think it has much if anything to do with lying, but that more people are willing to give an opinion to someone who comes to them, whether in the street, at the door or over the phone, than will leave home to vote or even send off a postal vote. They may also feel that a quick word to a pollster doesn't matter very much.
Most political parties know that they lose elections because of erstwhile supporters who don't vote, rather than those who switch to 'the other lot'.
--------------------
"He isn't Doctor Who, he's The Doctor"
(Paul Sinha, BBC)
Posts: 24276 | From: Newport, Wales | Registered: Apr 2004
| IP: Logged
|
|
M.
Ship's Spare Part
# 3291
|
Posted
OF COURSE I lie in opinion polls, surveys* etc. Doesn't everybody? I suppose I'd sort of assumed, if I thought anything, that people's lies cancel each other out.
M.
*though I don't do many face-to-face surveys these days, as I always end up rowing with the surveyor.
Posts: 2303 | From: Lurking in Surrey | Registered: Sep 2002
| IP: Logged
|
|
Alan Cresswell
Mad Scientist 先生
# 31
|
Posted
Polling organisations also have to adjust for selection bias, the people who on the phone or the street refuse to answer the questions. Plus the awkward gits who find good reasons not to be able to answer the questions and argue with pollster. "What do you mean? I want to say I'm concerned about immigration because current regulations are too restrictive and you don't have a box to tick for that?" I might be like M.
--------------------
Don't cling to a mistake just because you spent a lot of time making it.
Posts: 32413 | From: East Kilbride (Scotland) or 福島 | Registered: May 2001
| IP: Logged
|
|
deano
princess
# 12063
|
Posted
Some of you say pollsters correct for the bias. Here is a direct quite form the article...
quote:
Anyway, dishonest respondents are a serious problem for pollsters, especially in a situation such as an election, in which the quality of the sampling will be tested shortly afterwards by the actual result.
There is almost nothing that polling organisations can do about this. Online polling may give some weight to how people voted in previous elections, although they may also be lying or misremembering about that.
So it doesn't look like it can be solved easily.
Also the assumption in the article is that the bias is toward the left, which means it won't self-correct by an equal amount of false answers cancelling each other out.
--------------------
"The moral high ground is slowly being bombed to oblivion. " - Supermatelot
Posts: 2118 | From: Chesterfield | Registered: Nov 2006
| IP: Logged
|
|
LeRoc
Famous Dutch pirate
# 3216
|
Posted
quote:
deano: So it doesn't look like it can be solved easily.
Of course it can. Pollsters need only to look at the polls for previous elections and compare them with the actual results, not at what people remember about them.
If the effect you describe has led to a 3% bias to the left in the past, then all pollsters need to do is correct their predictions 3% to the right. Which is what they do.
--------------------
I know why God made the rhinoceros, it's because He couldn't see the rhinoceros, so He made the rhinoceros to be able to see it. (Clarice Lispector)
Posts: 9474 | From: Brazil / Africa | Registered: Aug 2002
| IP: Logged
|
|
Enoch
Shipmate
# 14322
|
Posted
quote:
Originally posted by deano:
... So it doesn't look like it can be solved easily. ...
Yes there is. Ban polls, either all the time or within X weeks of an election. I think some countries do that.
As for the more ethical question, to what extent do we think a person is under a moral obligation to answer a pollster truthfully?
I don't think I've ever been polled - which says something for the degree of credibility I attribute to them. I can only remember one person ever saying they had been. So I don't know what my reaction would be. I suspect I would politely say 'it's none of your business'. But the pollster has butted into the victim's life, uninvited, with a question the victim might regard as personal and intrusive. I don't think I have it in my heart to condemn someone who lies in that situation.
--------------------
Brexit wrexit - Sir Graham Watson
Posts: 7610 | From: Bristol UK(was European Green Capital 2015, now Ljubljana) | Registered: Nov 2008
| IP: Logged
|
|
Sioni Sais
Shipmate
# 5713
|
Posted
deano:
Apart from citing one election, the 1992 one in which John Major's Conservative Party defeated Neil Kinnock's Labour Party, where do you have evidence that there is an assumed bias to the Left? There's no mention of Mrs Thatcher in the article for a start and the ORB survey does not ask about political allegience either.
My feeling is that the winning side picks up more of the late undecided votes than does the losers. A glance through wikipedia and elsewhere suggests tht while the core voters are quite willing to trumpet their allegience, those who are on the edge and a little unsure hedge their bets and are less demonstrative.
[ 22. September 2014, 11:33: Message edited by: Sioni Sais ]
--------------------
"He isn't Doctor Who, he's The Doctor"
(Paul Sinha, BBC)
Posts: 24276 | From: Newport, Wales | Registered: Apr 2004
| IP: Logged
|
|
Barnabas62
Shipmate
# 9110
|
Posted
I guess the real issue is methodological. Surveys by questionnaire require well designed questions and properly controlled sampling techniques. Their interpretation requires a pretty high level of "smart".
This guy is smart. His 538 blog became essential reading in the 2012 Presidential Election. Not because he was a pollster, but because he developed an effective interpretative method of making sense of the wide and prolific range of polling information available. [He did a good job, for example, in demonstrating the "out of line" nature of Gallop polls and providing possible methodological reasons for this.]
I guess we may find it difficult to accept that samples based on a very small percentage of a population may actually be rather good at forecasting outcomes. It seems counter-intuitive that this should be so.
But the truth appears to be that with good sampling and questionnaire methods designed to check on consistency, sample polls are pretty good at doing this. The Scottish Referendum poll of polls was predicting about 53/47 on the eve of the referendum. The actual result, 55/45, looks to be within the normal range of sampling error. In this case, the polls got the result right, but underestimated the margin.
There is a "snake oil" element to all of this of course. A famous Yes Prime Minister episode pointed out that it was possible to design surveys to produce any result you wanted to achieve, simply by rigging the way the questions were asked. Also, I think professional pollsters do try to evaluate the lie and the secrecy factor, try to identify the extent to which it might bias results and make allowances for that.
A healthy scepticism, coupled with a critical approach to both methods and claims based on sample data seems pretty wise to me. But sample polling is here to stay, so it's a good idea to learn how to weigh the results critically, rather than just dismiss the whole enterprise out of hand as all "snake oil".
--------------------
Who is it that you seek? How then shall we live? How shall we sing the Lord's song in a strange land?
Posts: 21397 | From: Norfolk UK | Registered: Feb 2005
| IP: Logged
|
|
Schroedinger's cat
Ship's cool cat
# 64
|
Posted
If you are doing polling on questionnaires for academic purposes, then it is crucial to take into account the possibility of people lying, misremembering, exaggerating etc. There are a range of techniques for this, especially when the sample size if large, however these do require time and effort, and these are why most academic results give "indications" and often discuss the "significance" of results - just because a survey shows a bias towards one answer does not mean that this is the true answer - the result may not be statistically significant.
The problem is that political pollsters do not have the time to do this sort of analysis. their questions are much simpler, and they do, in some cases, take into account the significance of results, but this level of analysis is not possible when the results are needed for the late night news the same day as the poll is taken.
Do I lie? I try not to. If I was not going to give accurate information, I would refuse to do the poll. However often, a poll will polarise my position more than is accurate. I may say, for example, that I will vote Labour, if the choices are Labour or Conservative. However, that takes no account of the difference between a dyed-in-the-wool Labour supporter who would only not vote for them if they proposed wholesale murder of women and children, and the person who is just edging towards them, and may be switched by the smallest local thing.
The other problem is that they are often reported in a far more black-and-white way than is justified. They might show a "small victory" for one party, that is not statistically justified. But the news reporters don't like having to say "we have done surveys, but they have told us nothing significant".
--------------------
Blog
Music for your enjoyment
Lord may all my hard times be healing times
take out this broken heart and renew my mind.
Posts: 18859 | From: At the bottom of a deep dark well. | Registered: May 2001
| IP: Logged
|
|
Horseman Bree
Shipmate
# 5290
|
Posted
There's an element of uncertainty that has been introduced by the use of cellphones. A lot of people don't have landlines, preferring to use cellphones (or Skype and its equivalents) and therefore don't get direct calls. In North America, cellphone numbers are not listed, and there are "Do Not Call" registries, which are necessary because the phone user gets billed for airtime and doesn't want unwanted calls.
Also, even if they do get the calls, their location is not known to the caller, so they can't be included in analysis by seat/town/whatever. This all means that the uncertainty rate is higher than it used to be, and significant swaths, particularly of younger respondents, are simply not available.
Calls of "lies" are almost always from people who don't like the result, rarely from people who actually have any evidence.
[ 22. September 2014, 12:10: Message edited by: Horseman Bree ]
Posts: 5372 | From: more herring choker than bluenose | Registered: Dec 2003
| IP: Logged
|
|
tclune
Shipmate
# 7959
|
Posted
quote:
Originally posted by LeRoc:
IIRC, the polls predicted a small 'No' victory. What we got was ... a small 'No' victory. I'd say they were pretty much spot on.
[Tangent}In the US, a 10% margin of victory is considered a landslide. Is that only here? Are other countries generally more uniform in their views than we are?[/Tangent]
--Tom Clune
--------------------
This space left blank intentionally.
Posts: 8013 | From: Western MA | Registered: Jul 2004
| IP: Logged
|
|
Boogie
Boogie on down!
# 13538
|
Posted
quote:
Originally posted by M.:
OF COURSE I lie in opinion polls, surveys* etc. Doesn't everybody?
So do I - especially to canvassers. I say 'of course I will vote for you/your party' to all of them. It gets rid of them far quicker than arguing.
--------------------
Garden. Room. Walk
Posts: 13030 | From: Boogie Wonderland | Registered: Mar 2008
| IP: Logged
|
|
LeRoc
Famous Dutch pirate
# 3216
|
Posted
quote:
Horseman Bree: There's an element of uncertainty that has been introduced by the use of cellphones.
Normally, pollsters correct for that too.
quote:
tclune: In the US, a 10% margin of victory is considered a landslide. Is that only here?
Yes, that seems to be a US thing. It's quite remarkable actually, how close the two parties often are.
--------------------
I know why God made the rhinoceros, it's because He couldn't see the rhinoceros, so He made the rhinoceros to be able to see it. (Clarice Lispector)
Posts: 9474 | From: Brazil / Africa | Registered: Aug 2002
| IP: Logged
|
|
Enoch
Shipmate
# 14322
|
Posted
In the UK a 10% difference may not be a landslide but is significant enough to be decisive, not a small margin or a close run thing.
With a referendum, it's a simple headcount. With a general election, it's what translates into elected members. The first past the post system distorts that heavily. Depending on where the votes are and for whom, a small overall difference in headcount can translate into anything from a landslide to no overall control. In 1951, the Conservatives evicted Labour from government and got a small working majority despite getting a headcount that was c200,000 votes less than Labour.
--------------------
Brexit wrexit - Sir Graham Watson
Posts: 7610 | From: Bristol UK(was European Green Capital 2015, now Ljubljana) | Registered: Nov 2008
| IP: Logged
|
|
Heavenly Anarchist
Shipmate
# 13313
|
Posted
Online polls are generally taken from a panel of signed up members so that weighting is easier. This has it's own issues of bias but does mean that polling companies have a record of how you have previously answered questions and I'm sure they find the 'misremembering' fascinating. I'm on several online polling panels, the payment is very small for this type of panel (around 50p each one as opposed to a detailed consumer research company which pays £3 each) but I'm guessing a lot of correspondents are genuinely interested in polls and that helps with honestly. Interest in politics is what motivated me to join the panels.
The most exasperating question for me is the YouGov one asking me which political party I will vote for in the next general election. I answer according to the party the person I will vote for is in but I do not have an allegiance to his party so the question is inaccurate for me.
--------------------
'I love deadlines. I like the whooshing sound they make as they fly by.' Douglas Adams
Dog Activity Monitor
My shop
Posts: 2831 | From: Trumpington | Registered: Jan 2008
| IP: Logged
|
|
North East Quine
Curious beastie
# 13049
|
Posted
I'm signed up to YouGov too. I do try to answer questions honestly, and I do always answer questions about voting intentions scrupulously.
The recent YouGov ones had a flaw regarding voting history as they asked "which party". We can vote for more than one party for Holyrood, and as I do split my vote, I have a choice of truthful answers. ![[Biased]](wink.gif)
Posts: 6414 | From: North East Scotland | Registered: Oct 2007
| IP: Logged
|
|
Beeswax Altar
Shipmate
# 11644
|
Posted
Political opinion polls should be outlawed. I'm convinced politicians would run on what they truly believed and voters would vote for the candidate they truly preferred. Political coverage is too much about the horse race.
--------------------
Losing sleep is something you want to avoid, if possible.
-Og: King of Bashan
Posts: 8411 | From: By a large lake | Registered: Jul 2006
| IP: Logged
|
|
Barnabas62
Shipmate
# 9110
|
Posted
I've got some sympathy for the view that published polling may have some influence on results. The "bandwagon effect" is real, but I guess that the "uncertains" who jump on bandwagons need evidence to make them jump.
"Bandwagon effects" seem to be based more on what you might call "instant appeal" factors, rather than serious policy considerations.
But in a free society, that freedom includes the right to make a vote choice for non-serious or ephemeral reasons. My mum and dad nearly always voted differently. He was staunch Labour, would have voted Labour if the Conservative candidate was the Archangel Gabriel and the Labour Candidate was a donkey. He was a loyalist. My mum did different as an exercise of personal freedom. She was a classic nonconformist! Sure, they would argue politically, but in the end I think what influenced their votes most didn't have a lot to do with policies. I don't think they are that unusual in that.
Virtually all politics in the democratic West these days is strongly influenced by the concept of the floating voter. How are they to be influenced. And polling is the means of checking, dynamically, whether the electoral strategies in play are having any effect, and if so how much. In long election campaigns, like the Scottish referendum and the US Presidential campaigns, there is time to adapt the strategies of appeal.
My guess is that even if, by some (pretty unconstitutional) process, the publication of polls results was outlawed during election campaigns, the parties would still find ways of measuring bandwagon effects. Only we wouldn't get to know the results. And there may still be claims made that "the tide is turning in our favour".
So I don't think in practice there is much to be done to change the political desire to measure what's going on. Once that is accepted, the outlawing of polls looks more like suppression of information which a free electorate is entitled to know and make use of, if it suits particular electors' books.
--------------------
Who is it that you seek? How then shall we live? How shall we sing the Lord's song in a strange land?
Posts: 21397 | From: Norfolk UK | Registered: Feb 2005
| IP: Logged
|
|
Heavenly Anarchist
Shipmate
# 13313
|
Posted
One of my favourite bits of UK polling trivia is on referendums. YouGov's Anthony Wells (owner of UK Polling Report) says that if you poll the British public about whether they want a referendum on any subject they will always want one. Any subject, regardless of constitutional importance. Except for whether the monarchy should be abolished, they definitely don't want a referendum on that.
--------------------
'I love deadlines. I like the whooshing sound they make as they fly by.' Douglas Adams
Dog Activity Monitor
My shop
Posts: 2831 | From: Trumpington | Registered: Jan 2008
| IP: Logged
|
|
Beeswax Altar
Shipmate
# 11644
|
Posted
quote:
"Bandwagon effects" seem to be based more on what you might call "instant appeal" factors, rather than serious policy considerations.
But in a free society, that freedom includes the right to make a vote choice for non-serious or ephemeral reasons.
Which is why I favor amending the constitution to provide for election of the president via reality television show entitled the American President.
Other than meeting the constitutional requirements for eligibility, interested candidates must collect a certain number of signatures in each state and pay a fee. Then, let the game begin! Each candidate will get the opportunity to make a stump speech on television. Then, the voters vote and one or more candidates are eliminated. After that, the speeches will be over issues determined by the voters. Candidates give speech. Voters vote. Another campaign is suspended. Eventually, after the issues are addressed, the voters will have eliminated enough of the candidates to make a debate possible. Moderators and formats will be determined by the polls. The candidates debate. The voters vote. Another campaign is suspended. Eventually, only four will remain. Those four pick running mates. Running mates give speeches. Voters vote. Another campaign is suspended. You get the idea.
Given the US love of reality shows, voter participation should be very high. Obviously, some games will have to be introduced to mix things up. Identifying foreign leaders with the country they lead might be interesting. Maybe a presidential simulator of some sort could be invented.
--------------------
Losing sleep is something you want to avoid, if possible.
-Og: King of Bashan
Posts: 8411 | From: By a large lake | Registered: Jul 2006
| IP: Logged
|
|
Barnabas62
Shipmate
# 9110
|
Posted
Thanks BA; that made this Host smile!
(As did Marlee Matlin's rather good playing of the deaf pollster Joey Lucas in "The West Wing", which I've been re-watching recently).
A strong sense of the absurd is very helpful when discussing politics.
--------------------
Who is it that you seek? How then shall we live? How shall we sing the Lord's song in a strange land?
Posts: 21397 | From: Norfolk UK | Registered: Feb 2005
| IP: Logged
|
|
|
Printer-friendly view
