Even if you don’t look for a reason to dismiss a poll because you don’t like its findings, it can be surprising, even baffling, that a poll of just 1,000 people is supposed to be enough to judge views. from a whole country.
How can that be? The math behind sampling is old, well established, and simple enough to be taught in schools. But if you’re not familiar with it, in my experience it’s usually a more intuitive explanation you’re looking for rather than a step-by-step explanation of detailed math.
Pioneer depth sounder George gallup had a simple and powerful analogy. Imagine that you have a bowl of soup in front of you. How much soup do you need to taste before you know how the whole bowl will taste? Very little. Less than a full spoonful, even.
Of course, you have to swirl the soup, especially if it has chunks in it (the soup equivalent of a random sample). But then you can take just a small amount of the soup and make a sure prediction of how the whole bowl will taste.
Adolphe Quetelet, the 19th-century inventor of the Body Mass Index (BMI) and the concept of the ‘average person’, had a more upscale version of it, although it was not developed specifically on polls:
Do I have to drink the whole bottle to judge the quality of the wine?
Back to the soup, now imagine replacing the soup bowl with a giant soup tub. Again, vigorous swirling may be necessary. But again, you will be sure to know what soup looks like from the small proportion you eat. Whether it’s a bowl or a tub of soup you taste, it doesn’t make much of a difference in how much you need
eat to be sure you know what the soup looks like.
Therefore, that sample of around 1,000 is sufficient whether you survey, say, the UK or the US with its much larger population. Having a lot more soup in your receptacle (the locals) doesn’t require you to taste a lot more soup (poll a lot more people) to experience the taste (the audience’s opinion).
It’s the same with medicine. A blood test takes a tiny fraction of your blood, but a few drops are enough. The medical staff never say “we will take double the usual amount of blood today to be really sure of the results.”
Therefore, just as you don’t care how little blood is drawn from you for the test, you shouldn’t worry about a survey of “only” 1000 people. There is much more to doing correctly in a survey. But when it is done well, it is indeed sufficient to measure the opinions of the country.
To learn more about how opinion polls work, including what the margin of error on polls actually means, check out my new book in April, Polling Unpacked: The History, Uses and Abuses of Political Opinion Polls.