Right now, in a room somewhere in London, five men are trying to work out how the UK is voting. Most people haven’t voted yet, but they’re already figuring out how we all will.
This isn’t just a poll, it’s an exit poll – the exit poll. In less than 9 hours, it will set Twitter ablaze and pundits in a four-hour frenzy until real results start pouring in.
An exit poll works by asking people how they have voted, rather than how they will vote. People have been dispatched to 140 polling stations around the UK, stations thought to be representative of the constituency they’re in. Every nth person who turns up at one of these polling stations, say every 10th, is then asked how they voted as they leave.
These answers, from those who do answer, are collected and sent back to the exit poll room. Almost no one knows where this room is. The five men stowed away inside it then start to analyse the results, looking for patterns. By now they should have just received their first batch of real, actual voting data.
Right now, five men are trying to work out how the UK is voting.
Data will come in thoughout the rest of the day in ‘loads’ or ‘drops’. A final one will come in at 9.15pm, just half an hour before the exit poll has to be released to broadcasters, and 45 minutes before it’s released to the world. If that final drop of data doesn’t fit with everything else the exit pollers have seen, they will have to rapidly re-write their model.
Something like this happened in 1992. The exit poll had been handed to the BBC (it is now run jointly by the BBC, ITN and Sky) and was ready to be released. Then, a few minutes before David Dimbleby was due to announce it on air, he was suddenly told the poll – which had predicted a Labour majority – was changing. As he went live, he didn’t have a poll to announce.
The first time he saw the new poll was when it flashed up on his screen at 10pm. It showed a hung parliament. The exit poll had belatedly picked up some of the shy Tories who would hand John Major that most unexpected of election victories.
Will something like this happen again this year? The 2015 election is the most complicated forecasts have ever tried to predict.
The five men responsible for the poll have to come up with a model on the fly.
And all the usual time pressures apply. The five men responsible for the poll have to come up with a model on the fly, with data being handed to them in drips. They have to latch onto things that they think they might be seeing – like a strong first-time incumbency effect for Tory MPs, or an outsized swing against Labour in its Scottish heartlands – and rapidly test whether they can be applied across the UK.
“There are a lot of variables flying around,” says Steve Fisher, one of the five men. Fisher is an Oxford academic behind Elections Etc, one of the election forecasts long tracked by May2015. He is one of the three most senior people in the room. He and Jon Mellon, a research fellow at Nuffield College, Oxford, have to come up with the exit poll’s statistical model over the next 9 hours.
If that model is wrong, and the exit poll is way off tonight, “We’ll be set back another 30 years”, says Sam Woodhouse, the BBC’s election night editor. The poll has built a stellar reputation since the ’92 debacle. It was close enough in ’97 – accurately predicting a Labour landslide – and very close in ’01, ’05 and 2010, when it correctly predicted the Lib Dems would win fewer seats than they did in ‘05. But one bad poll will ruin an string of good ones, however long.
The ultimate judge of how accurate the model is will be John Curtice, the UK’s pre-eminent psephologist. Curtice has been in exit poll rooms since the ‘70s. He was there in ’92. He effectively took over from David Butler, his former supervisor at Nuffield, and the man who explained British elections on TV from their inception in the 1950s through to Thatcher’s victory in 1979.
The indefatigable Butler, who first went on air at 26 and is now 90, will be at the BBC tonight, but he no longer has Curtice’s responsibility. As Fisher put it to us, “Ultimately if John isn’t happy, we have to look again.” The room is democratic, but Curtice must sign off on decisions. Fisher, Mellon and he will be helped by Rob Ford, the co-author of the set text on Ukip – Revolt on the Right – and a lecturer at Manchester.
The ultimate judge of how accurate the model is will be John Curtice.
Ford’s role is closer to background analysis and explanation than specific modeling. It’s a role Fisher played in 2010, his fourth time in the exit poll room. In 1997 he was ‘little more than a tea boy’. That supporting role is being played this year by a PhD student of Rob’s, Patrick English. In thirty years he may be in charge. There seems to be a decades-long hierarchy to the room.
Two other men will be on hand, Colin Railings and Michael Thrasher, who are there more as questioning emissaries from ITV and Sky than forecasters themselves. Together, this band will spend the rest of the day giving us a result six hours earlier than we’ll otherwise be able to start guessing it.
How exactly they do it is complicated. It involves statistical testing, ‘empirical priors’ and ‘theoretical priors’. But put simply, if forecasters start to see things they expect – like a strong incumbency effect for Lib Dems, or a lower Ukip vote than they once envisioned – they are more likely to grasp onto that variable and include it in their model.
Modelling change is harder when you have to add new polling stations.
Once they are happy with their variables they will start to extrapolate, and come to the seat prediction that will flash up onto our TV screens at 10. The critical point is that they don’t try and predict an overall vote share. Instead, they try to work out change since the last election. That way, any bias in the exit poll method should cancel out across elections.
Modelling change is harder when you have to add new polling stations. They have had to this year so they can model Ukip and the SNP. They have clipboard-holders in Scotland and across eastern England today, whose data they have no comparisons for. That makes their task all the more fraught.
How accurate do they expect to be? Fisher hopes to be within 20-25 seats on the main two parties, a similar range to the one offered by pre-election forecasters like Newsnight’s Election Forecast.
But this year that is unlikely to be thought good enough. If the exit pollers are out by just a few seats, they could get the largest party the wrong way round, or predict an anti-Tory majority when one never materialises. In 1997, a 20-seat miss didn’t matter. This year it does. The exit poll has to be as accurate as it was in 2010 to satisfying the baying hordes that await it online.
Thankfully, we should get a better idea of its accuracy early in the night than we did in 2010. Last time round the BBC spent three hours rubbishing its own exit poll after the first seats to declare, in Sunderland, showed unexpected swings. The people that had come up with the poll – Curtice, Fisher and co – were unperturbed. They reportedly took one look at the Lib Dem share and put their feet up.
That wasn’t how it was reported on the BBC. This time Woodhouse, the BBC editor, is going to have a more direct line to Curtice, and put him on air. If you want to know how accurate the exit pollers think their forecast is, listen to Curtice. Only one important result is due before 2am – Nuneaton, a seat Labour need to win, at 1.30 – but if Curtice is confident in his poll by then, take note.
He and his team are unlikely to very confident in their prediction until later in the night. In 2010 they knew their model was right when they correctly forecast Tooting, which traditional swing models didn’t expect Sadiq Khan to hold. There are likely to be similar seats tonight. We will try to explain them as they come in, but Curtice has been doing this for 35 years. We’ll be listening to him.