What is the margin of error in a poll?
It would be virtually impossible to conduct a poll on the entire voting population in the United States. Pollsters therefore question a sample of a population. For example, a poll on voter sentiment might report how a group of randomly selected registered voters — not all voters — feel about the president. The margin of error in a poll is a measurement of how accurately the results of the poll reflect the “true” sentiment of the whole population. It is a statistical formulation of how well the sample (those voters who took the poll) reflects the general population (all voters).
If people are selected randomly and some basic statistical assumptions are made, there is a mathematical formula for the margin of error. The way to understand a poll result such as “49 percent of American voters prefer Bush to Kerry, with a margin of error of three percent” is that 49 percent of those questioned prefer Bush to Kerry. If the poll were conducted again (with a different sample of the population), the percentage of those questioned preferring Bush over Kerry is likely to be a value from 46 (49 minus 3) to 52 (49 plus 3). How likely? The margin of error is calculated so that a new poll’s result is 95 percent likely to lie within the margin of error. If a hundred similar polls were conducted, we would expect that in 95 of them, 46 to 52 percent of people would support Bush over Kerry.
The margin of error can be made smaller by polling more people. A poll involving only a few hundred people may have a large margin of error, and a poll with thousands of people will have a small margin of error. Political polls by professional polling agencies are designed to poll just enough people to get a somewhat small margin of error (usually two or three percent). To poll more people would be expensive, and to poll fewer people would mean less accurate results.
Of course, we aren’t really interested in a sample of the population; we want to know the voter sentiment in the whole population. The margin of error tells us that there is a 95 percent certainty that the sample does reflect the whole population. The percentage of all people supporting Bush over Kerry (also called the “true value”) is therefore most likely to be between 46 percent and 52 percent. However, the best estimate of the true value is 49 percent.
Why is this so important? The results of a poll may tell one story without the margin of error, and another with it. For example, if Kerry is leading the polls against Bush by 51 percent to 49 percent, one might think this bodes well for Kerry. However, if there is a three percent margin of error, then there is a substantial chance that the population as a whole supports Kerry as little as 48 percent, and Bush as much as 52 percent. A more accurate description of the ostensible Kerry lead is that the two candidates are neck in neck.
Of course, all of this assumes that the poll talked to a sample of people representative of the whole population, a subject of some controversy.