What was wrong with polling methodology?

A lot of effort goes into market research, and attempting to ensure that opinion polls are balanced and representative. That being so, let's compare the BBC "Poll of Polls"(which gathers together the results of a basket of opinion polls) with the final percentages who voted:

Party	Poll of polls/%	Actually polled/%	Difference (nearest 0.5)/%
Conservative	34	36.9	3.0
Labour	33	30.4	-1.5
UKIP	14	12.6	-1.5
LibDem	8	7.9	0
Green	6	3.8	-2

An interesting phenomenon was that the BBC exit poll was very, very close to being dead on - certainly it picked up the fact that the Liberals were going to be hammered, SNP were going to sweep the board in Scotland and the Conservatives would have roughly an overall majority, whereas modelling on the basis of the Poll of Polls pretty consistently came back with a hung parliament and the LibDems not doing so badly. It's hard to overstate the sense of shock and disbelief that the exit poll created, but as a reminder ... Paddy Ashdown said he'd eat his hat.

The difference between the advance opinion polls and the actual outcome was enough to totally change the shape of the parliament. So how come this big discrepancy?

It may be that people's statement of intentions in opinion polls was not reliable. Or it may be that the undecideds didn't distribute their votes evenly when it actually came to casting a vote. My hunch, however, is that there is a social phenomenon involved as well.

Let's hypothesise that there is a block of voters that the opinion polls don't reach, and that the voting intentions of this block of voters aren't congruent with those of the opinion polls. Is it possible to come up with a size for this block and a distribution of their voting intentions which, when you include these voters in with those people reached by the opinion polls, gives you a final distribution of votes that matches what was seen? The answer to that is, yes. Suppose that the opinion polls are actually only able to reach 5/8 of voters, and the other 3/8 for whatever reason are invisible to the opinion pollsters. Then supposing that the distribution of the votes in this group is completely different from that of the 5/8...

	PoP	Actual share in election		Share of invisible votes		... results in this share
Conservative	34	36.9		42		37.0
Labour	33	30.4		26		30.4
LibDem	8	7.9		8		8.0
UKIP	14	12.6		10		12.5
Green	6	3.8		0		3.8
Other	5	8.4		14		8.4
	100	100		100		100.0
			Invisible votes/%
			60

This is a pretty contrived option, obviously - the idea that opinion pollsters are failing to reach 60% or more of the population is pretty implausible. But the principle is solid. To get to an invisible share of 60%, I worked on the biggest proportional drop - the Poll of Polls figure of 6% for Greens and the actual vote of 3.8%. Suppose instead that change of voting intentions on the day means that of the 6% of people who said reported in the PoP that they would vote for the Greens, only 5% actually did, the other 1% switching to "Other". We now only need a block of invisible voters that is half the size:

	Poll of Polls	Actual share in election		Share of invisible votes		... results in this share
Con	34	36.9		47		37.0
Lab	33	30.4		21		30.2
Lib	8	7.9		8		8.0
UKIP	14	12.6		8		12.6
Green	5	3.8		0		3.8
other	6	8.4		16		8.3
	100	100		100		100.0
			Invisible votes/%
			30

What becomes apparent is that the smaller the group of invisible voters is, the larger the proportion of them that vote for the Conservatives. And this does have a correspondence with an aspect of the real world. The more "conservative" - self-sufficient, independent and autonomous - someone is, the less likely they are to be involved in the rest of society. Their phone number is not accessible as they use the telephone preference service. They get their groceries and so on delivered, rather than going into town to get them. The choices of the Conservative consumer are more likely to result in them being invisible to any of the means that opinion pollsters have available at their disposal to ask for their opinion. Possibly they are also more private and reticent about sharing their views as well.

My hunch is that the size of this block of "invisible votes" is actually quite a lot smaller than 30%, but that there is a growing section of the community who behaves in this way, and who opinion poll organisations are failing to reach. This is just one of the factors on top of others which resulted in the discrepancy between opinion polls and the final outcome of the election. But I think that there may be a significant methodological issue here for opinion polling.