Are machines the new pollsters?

Donald Trump’s win earlier this month was not only a defeat for Hillary Clinton, but a blow to the reputation of many polling organisations which projected Clinton’s lead right up until election day. While most of the media on both sides of the Atlantic took the polls as credible and reported as such, Trump, in typical antagonistic style, was not convinced:

“I do think a lot of the polls are purposefully wrong. I think I can almost tell you by the people that do it,” he said. “The media is very dishonest, extremely dishonest. And I think a lot of the polls are phony. I don’t even think they interview people.”

Phony and dishonest? Unlikely. Wrong? To a large extent, yes.

From 4^th to 11^th November a poll by YouGov and The Economist put the national polling averages for Clinton and Trump at 45% and 41%, respectively. Bloomberg/Selzer put the averages at 46% and 43%. Similar inaccuracies in predicting the outcome happened in the lead-up to the Brexit vote. On the evening before the referendum polling day, a number of polling companies predicted a narrow victory in favour of the Remain campaign.

Going back further, the same thing happened in the 2015 UK General Election, when polling companies predicted a hung parliament, but the Conservatives surprisingly won an overall majority. Why is it that these polls – which purport to reflect public sentiment and make considerable profits based on this – get things so wrong? Could it be that polling is just not an accurate way of capturing public opinion?

Despite the sensitive nature of political polling, and polling on such divisive characters as Trump and Clinton, it’s not impossible to accurately predict how people are going to vote. While the majority of polling companies were incorrect with their predictions, it was in fact a machine – powered by artificial intelligence – that said Trump would become President-elect.

Politics is about more than data and numbers, it is about emotion, circumstance, and climate. Has the US election shown that perhaps a machine knows us better than we know ourselves? Could AI play a more significant role in the way polling and research companies gather their intelligence and data in the future?