November has thus far seen considerable variation in the polling fortunes of the Liberal Democrats.
Their standing has veered from 11 per cent with ICM and 10 per cent with Ashcroft down to 6 per cent with YouGov and Survation and a twenty-four year low of 5 per cent with Opinium, while the May2015 average stood at 7.7 per cent shortly thereafter. What explains the differences?
All polls are subject to sampling error – the statistical variation that the same pollster would encounter between different random samples of the same population. But for a party polling 7.7 per cent, a sample of 1,000 has a margin of error of just ±1.65 points – all of these polls diverge from the average by more than that.
In most cases, this divergence can be attributed to “house effects”, the difference in the results that two pollsters would obtain from the same sample, resulting from differences in the methodologies, and specifically the way they weight their samples. The key difference in this case is the treatment of those who say they would vote, but don’t know or refuse to say who for.
YouGov excludes them altogether and reports only those who state a voting intention. Since those that “don’t know” include a disproportionate number of 2010 Lib Dem voters, this depresses the YouGov figures relative to pollsters that make some allowance for them.
Spiral of silence
ICM, by contrast, uses the composition of those that don’t know or won’t say to adjust their numbers for what is known as the spiral of silence. This technique was one it pioneered in the 1990s, following the industry-wide debacle at the 1992 election, when polls were about 9 points wrong on average.
Subsequent investigations revealed that many “don’t know” respondents had actually voted Conservative in the prior election. To counteract these “shy Tories”, ICM now reallocates 50 per cent of those not stating a voting intention to the party they voted for in 2010.
Recent ICM surveys have had a significant proportion of don’t know/refused among 2010 Lib Dem voters (e.g. 34 per cent in this recent one), magnifying the adjustment to around 3 points. While this is just one of many methodological differences (ICM is by telephone, YouGov is online), it does explain nearly all of the 3 to 3.5 points by which the two pollsters have typically differed in recent months.
Lord Ashcroft uses this ICM technique as well, however his national poll shows very few “Don’t knows”, so the adjustment is very small. Instead, his poll has a relatively high level of Lib Dem support before the adjustment. There is no obvious reason for this, but it’s notable that Ashcroft National Polls typically have among the lowest support for the Conservatives and Labour and higher figures for everyone else, including the Lib Dems.
The record-low Opinium poll last week was interesting because it isn’t due to house effects.
Survation also performs a similar adjustment to ICM, but with 30, rather than 50, per cent of “undecided” and “refused” reallocated. However this is offset by the fact that Survation’s polls, which include UKIP in their main prompt, typically have by far the highest UKIP polling – 23% in their latest poll, including 27% of 2010 Lib Dems, compared to 12 per cent of 2010 Lib Dems in the YouGov poll – more than enough to offset the reallocation.
The record-low Opinium poll last week was interesting because it isn’t due to house effects – its polling, on average, shows very similar Lib Dem support to the industry average. Instead, the 5 per cent reading is four points lower than the more typical 9 per cent it reported 10 days earlier. Like all pollsters, Opinium weights responses by demographics, but unusually, they don’t weight either by past vote or party identification.
Demographic weighting does iron out some of the response bias that pollsters encounter, to the extent that party support differs across various demographic groups. But if differential response arises within a particular demographic, that feeds through into the results. That may explain the sudden Lib Dem drop on this occasion, but since Opinium don’t ask (or at least, don’t publish) a past voting breakdown, we can’t say for certain.
Of course, we don’t know which approach provides the most accurate snapshot of how people would vote in an election, but we can see that the snapshots are quite different.
Studying specific seats
Turning to seat projections, on a uniform national swing, the May2015 polling averages would give the Lib Dems 23 MPs. But with such a substantial swing against them, the uniformity of swings around Great Britain bears scrutiny. As May2015 detailed last week, the Lib Dem-to-Labour swing is far greater than the Lib Dem-to-Conservative swing.
By examining the constituency voting question in the most recent Lib Dem battleground Ashcroft poll, we can see that the former swing is some 1.5 to 2 per cent less than in the Ashcroft National Polls and the latter 2.5 to 3 per cent more than nationally. I’ve excluded Eastleigh from these calculations, due to the risk that the 2013 by-election (and disproportionate campaigning) in the Hampshire constituency has made it unrepresentative of swings in other Con-Lib contests.
But it’s not just the party in second place that incumbents should fear. In Scotland the national SNP surge is so great that if repeated uniformly across Scotland, it would unseat an additional 2 Lib Dem MPs in seats where the SNP wasn’t even runner-up in 2010, and come close to unseating a third.
It would also change the winner of several seats that I already have the Lib Dems losing. Given the size of the moves, uniformity is even less likely across Scotland than south of the border, which puts all the more meaning on Lord Ashcroft’s inaugural Scottish constituency polls, due next month.
In terms of Lib Dem seats where nationalist parties actually came second in 2010, Gordon looks very likely to fall to the SNP. In Wales, Ceredigion will likely be more of a battle, but none of the (admittedly limited) Welsh polling thus far suggests sufficient gains by Plaid Cymru to win it, so I’ve marked it as a Lib Dem hold.
The May2015 averages show national swings against the Lib Dems of 9.5 per cent to Labour and 5.5 per cent to the Conservatives. Applying the differentials above (and dividing them by 2), these become 12.2 per cent and 3.9 per cent respectively. Applying these differential swings (and the SNP adjustment) across the battlegrounds saves the Lib Dems 5 seats in contests with the Conservatives but costs them 2 in defences from Labour and 2 to the SNP that they would otherwise have held, resulting in a net saving of one seat.
Applying differential swings across the battlegrounds saves the Lib Dems 5 seats against Tories but costs them 2 each to Labour and the SNP.
Why use Lord Ashcroft’s constituency polling in this way instead of just taking the results at face value, given that individual seats often behave differently to the national or even battleground picture? First of all, there are very few pollsters (in most cases, just one) polling each constituency, so the impact of those pollsters’ house effects is far greater than in national polls.
As mentioned above, Ashcroft polling tends to show lower-than-average support levels for Labour and the Conservatives and relatively high figures for the Lib Dems. This is why the 3.9 per cent swing to the Tories that I’m working with is larger than the 2 per cent taken directly from the poll. Comparing constituency polls with national polls by the same pollster “nets out” these effects to a large extent, save for any “in-house” house effect between national and constituency polling.
Secondly, constituency polling tends to be published with a lag, as the various constituencies aren’t all polled concurrently. Things have changed in national polling since the fieldwork took place in August and September, with Labour dropping several points. This is enough to offset the Lib-Lab Ashcroft house effect, so in Lib-Lab seats my 12.2 per cent swing is basically the same as in the seat polls.
Thirdly, sample size is an issue. The Ashcroft battleground polls collectively have a massive sample (22,000 in the latest), but in individual constituencies it is only 1,000, as in national polls.
The margin of error for the spread between two parties on around 32 per cent of the vote is about ±1 point for the full sample or ±2 points for the 5 LIB-LAB contests, but ±5 points for each individual seat’s thousand-voter sample.
Of the 22 seats polled, 12 had majorities smaller than that, a further 7 had statistically significant majorities but the aggregate swing correctly predicted the winner, while the 3 “surprises” split 2-1 between holds and losses respectively.
So while constituency polls provide us with extremely useful data, they are still opinion polls and there are statistical benefits to aggregating them, particularly where we are looking at overall seat numbers rather than specific constituencies. And as Lord Ashcroft himself says repeatedly, polls are snapshots, not predictions.
We need to make some further adjustments, to reflect the likely outperformance of the Lib Dem MPs newly elected in the previous election, together with the likely underperformance of Lib Dem candidates replacing those standing down this time. As May2015 has reported, Ashcroft’s polls show this incumbency effect strongly (and only) applies in the case of Lib Dems, which echoes academic research.
For example Tim Smith’s work has estimated the incumbency effect at each British election since 1983. I prefer to exclude 1983, due to the likely distortions caused by the mass defection of incumbents to the then Liberal/Social Democratic Party Alliance from other parties (mostly Labour). From 1987 to 2010, the advantage to the Lib Dems of having an incumbent seeking re-election was about 10 points net (5 per cent swing) over both Conservative and Labour challengers.
Using a 10 point adjustment would, on these projections, save the Lib Dems 4 seats net from Tory challengers. It wouldn’t change any LIB-LAB contests, because all of the affected seats would be lost, with or without the adjustment.
Thus we arrive at a “nowcast” of 28 holds, 12 losses to Labour, 10 losses to the Conservatives and 7 losses to the SNP, including 4 seats where the SNP plus another party relegate the Lib Dems to third place.
Some have gone further, suggesting that Lib Dem incumbents might outperform a substantial swing against them. The evidence on this is mixed. In 2010, the Lib Dem vote share in seats being defended by non-first time incumbents actually declined by 1.1 points whereas it increased by 1.3 points nationally. About a point of that disparity can be explained by the fact that the Lib Dems, no longer led by Charles Kennedy, underperformed markedly in Scotland (where they were defending 10 of their 62 seats), seemingly to the benefit of Labour, then led by Gordon Brown.
We arrive at a “nowcast” of 28 holds – 12 losses to Labour, 10 losses to the Conservatives and 7 losses to the SNP.
The rest is harder to explain. It could be that the Lib Dems’ national vote increase came from anti-Tory voters who had previously voted Labour in non-Lib Dem seats, but who would already have been voting Lib Dem in Lib Dem seats. Normally polling data would be useful here, but unfortunately the polls can’t help in this case, because in aggregate they showed a Lib Dem national gain about three times larger than actually transpired.
What about well-known, long-serving incumbents? I’ve taken the 54 seats that Lib Dem incumbents defended in 2010 and grouped them by their year of first election to the Commons. For the reasons above, I’ve separated Scotland from the rest of Great Britain. We can see that the class of 2005 outperform their seniors, as expected. But among the other cohorts, there is no pattern to suggest a seniority bias, they all underperform the national vote change by similar amounts.
How well this would hold in the face of an overall collapse on the scale the polls currently suggest is hard to say. History tells us very little – the Liberal Democrats have never dropped more than 5 points in a single electoral cycle, and the original Liberal Party last did so in 1951.
In the Ashcroft poll, Simon Hughes (first elected 1983) in Bermondsey significantly outperforms the rest of the Lib-Lab battleground. But in contests with the Conservatives, the only statistically significant outperformance by a long-standing MP is in Sutton and Cheam, where Paul Burstow (1997) has managed to turn the swing on its head, achieving a 7 per cent swing away from the Tories. Elsewhere in the poll there is no evidence of such a trend.
So while having a large “personal vote” may matter when an MP retires, it is less clear that longer-serving MPs are better able to withstand a collapse in the national vote.
Matt Singh runs Number Cruncher Politics, a non-partisan rolling psephology blog, covering statistical analysis, opinion polls and politics.