Opinion Polling has achieved a certain notoriety in recent years. Its lowest point came following 2015 general election when it failed to ‘predict’ the correct outcome for that year’s particular election.
Or did it? Firstly polling is not a prediction, as Prof. Roger Scully repeatedly says but merely a snapshot of people’s attitudes and opinions based on specific questions asked. Secondly the polling was accurate for the total number of votes cast for each party. Where it failed was to identify the unique swings in individual seats.
Polling can be a remarkably good indicator of the broader picture.
They were pretty much on the money for both the Scottish and EU referendums for instance.
What helped in both cases was that the events they were talking about didn’t ask someone to imagine a possible future scenario. The situation and the choices were real. Each polling company used the exact words, or very similar, to those which were to be used on polling day.
They were also posed when there was a public discussion about both prospects, the media were committed, and the ‘national’ narrative was pushed by both particular issues. Much like polling day people are not expected to give an opinion in a vacuum, they make an informed decision.
Needless to say the closer to polling day, as the information flows and permeates through society, the more accurate the results.
At such times polling companies come to their own, and justify their existence.
Where it gets particularly difficult is when they are posing questions which are not on the political agenda. At such times the questions posed exist in a vacuum of information. There will undoubtedly be a percentage of the society who are committed or obsessed by a particular cause, one way or the other, which is reflected to a lesser or greater degree in the polling results. But for most the issue is not pressing and their minds aren’t focussed on it. In the absence of a concerted public debate, which fully engages the media and occupies the political agenda, it is unlikely that much could be read into the results of such opinion polls, other than to tell a section of society that they need to do more work to promote their cause.
As well as this, the wording of any question can also have an influence on the result. Pollsters can unintentionally direct those polled by the form of words they choose.
Let’s look at the Scottish Independence Referendum, 2014.
The prospects of an Independent Scotland has looked pretty bleak over the decades.
Throughout the 1980s and 1990s support for Scottish Independence bobbed around the low to mid 20s.
In April 2007 YouGov asked:
“If there were a referendum on whether to retain the Scottish Parliament and Executive in more or less their present form or to establish Scotland as a completely separate state outside the United Kingdom but inside the European Union, how would you vote?”
25% supported this statement, “In favour of a completely separate state outside the UK.”
However also in 2007 a Progressive Scottish Opinion survey asked,
notice that in the second opinion poll there is no mention of the words ‘separate’ or ‘UK’.
In 2009 ICM asked,
Only 20% supported, “Scotland should become a fully independent country, separate from the rest of the UK” – notice once again the words ‘separate’ and ‘UK’.
The same company, during the very same year also asked this,
This time 38% said, “I agree Scotland should become an independent country”.
The form of wording, both the written question and proposed answer, has an effect on the result.
But the biggest influencer on opinion poll results is the context in which it is asked.
The SNP promised an independence referendum in their 2011 manifesto. In all honesty they probably weren’t expecting that they would have to hold such a referendum. They Governed with a minority Government in the previous Parliament, and the electoral mechanics for Holyrood were designed to stop any one party from winning outright, thus forcing parties to compromise on manifesto pledges.
Things didn’t quite work out that way!
The SNP won a stunning victory 2011, winning an outright majority against all the odds. Their manifesto pledges would have to be honoured. Suddenly the prospect of Independence became very real indeed.
Posed with the exact same wording,
“Scotland should become a fully independent country, separate from the rest of the UK”, this time 27% said they supported independence ‘Separate from the rest of the UK’. A 7% increase.
The independence narrative was having an immediate impact.
Then again in August 2011, 35% agreed with
“(do) you agree or disagree with a proposal to extend the powers of the Scottish Parliament to enable Scotland to become an independent country, separate from the UK. If the referendum was held tomorrow, would you vote to agree or disagree with this proposal?”
In 2011/2012, Ipsos/Mori altered their wording to include the Devo/Max option. This reflected the narrative of the day. They conducted two opinion polls with the following wording,
“Thinking about Scotland’s constitutional future, which of the following statements most closely matches your views on this issue?”
This resulted in a drop in the support for independence. It was no longer a binary choice. People were given multiple choices, including increased power to the Scottish Parliament.
The “Scotland should become a fully independent country, separate from the rest of the UK” response received 27% of the vote on both occasions.
Also, on both occasions, the “Scotland should remain part of the UK with the increased powers outlined in the Devo Plus proposals” received 41% support.
Independence remained a minority sport, because the option of DevoMax – increasing Holyrood’s powers – seemed to be the moderate choice.
DevoMax was the most attractive option for Scottish voters.
In October 2012, however, David Cameron refused to allow the Scottish Government a DevoMax option. It then became a binary choice once again. This coupled with the Scottish Government’s launch of their White Paper brought the whole thing home to the people of Scotland. It was real. It was happening. They had to consider both options and come to a decision.
Because of the lack of media plurality in Wales, the BBC have a practical monopoly on news provision here. Consequently their annual St David’s Day opinion poll receives far more attention than any other poll in Wales. These polls are conducted by ICM, and provide the following wording,
“Wales should become Independent, separate from the UK”.
Again , notice the wording, ‘separate from the UK’.
The response to this ranges from a 3% low (immediately following the Scottish Referendum) to a 6% high (2015 – 2017).
That 3% received much press attention, and has been used as a stick to hit the independence movement in Wales ever since.
If you’re reading this you probably know this already.
Which brings me back to wording.
The Wales Governance Centre/ITV use YouGov and conduct far more regular opinion polls. In 2014 they provided the following wording: “‘Should Wales be an independent country?” note that this omits the ‘separate from the UK’ part.
Also more often than not, the Independence option is part of a multitude of choices. In each case there is large and sustained support for the Assembly to gain more powers.
In 2012 YouGov asked,
“which ONE, if any, of the following do you think should have the biggest say in how Wales is run?”
A massive 57% said that they believed that the Welsh Government and National Assembly should have the biggest say in how Wales is run. How many of you knew that?!
One poll which provided a starkly different form of words from the wording normally found in Welsh opinion polls was the one found in the July 2016 opinion poll by the Wales Governance Centre/ITV conducted again by YouGov. This time it posed three different question, asking the respondent to imagine different scenarios. When asked to imagine a scenario where the rest of the UK left the EU but an independent Wales got to remain, 28% supported that idea. Clearly, wording and context are everything.
Like Wales, the Scottish polls leading up to 2011/12 showed a massive support for increased powers to Scotland, but not independence.
There is immense good will towards our National Assembly and every time the question is asked if they should receive more powers the overwhelming response is yes.
People clearly want to see Wales gain more powers.
But it’s clear that context, and to a lesser degree wording, is everything.
As the prospect of Scottish Independence became clear; as the issues were increasingly debated in the public sphere; and as the wording of the actual question was provided, more and more people flocked to support Independence. This in turn gave the Scottish people confidence, forced them to ask difficult questions of themselves about their political beliefs and allegiances, and opened their eyes to the true relationship between Scotland and Westminster.
While we in Wales have no prospect in the immediate future of an Independence referendum, and therefore no focus, we can raise the level of debate; we can force the issue on the political agenda. The first step towards sovereignty and national self-determination is to talk about independence. We need people to be thinking about it and talking about it.
It needs to be part of our political discourse.
We’ll be accused of having constitutional obsessions. That there are more pressing matters to hand. These are all tactics to deflect us from the debate.
The rotten establishment wants to silence that debate and deny us space for the discussion that we need.
Much like totalitarian establishments over the centuries have attempted to deny the people the ability to read and share information, the British establishment also wants to deny us that space to share information about how an independent Wales would thrive and create wealth for her people.
Don’t be afraid. We don’t have all the answers yet, of course not. But by discussing it and opening people’s mind, together we are certain to find the answers.
Please feel free to share any material on this website on your friends’ social media pages. You never know, it might get them thinking.
I forgot to mention earlier that Alex Salmond and his team had learnt from previous polling. The purposefully omitted the words ‘separate’ and ‘UK’ from the question and answers posed on the referendum paper in September 2014.
The original question was, “Do you agree that Scotland should be an independent country?”. This was refused. The final text was, “Should Scotland be an independent country?”. Salmond and his team understood the power of words, the power of a positive response, and that framing the question correctly would impact on the result.