MARGIN OF ERROR
What is the Margin of Error in a Poll?
It would be virtually impossible to conduct a poll on the entire voting population in the United States. Pollsters therefore question a sample of a population. For example, a poll on voter sentiment might report how a group of randomly selected registered voters feel about the president. The margin of error in a poll is a measurement of how accurately the results of the poll reflect the “true” sentiment of the whole population. The margin of error cannot tell us about bias in a poll, whether people are telling the truth in a poll, or whether the people chosen for the poll are representative of the whole population. For example, if a poll on presidents is conducted at a local college, we are probably not going to have a representative sample of the American voting public.
The margin of error is a statistical formulation of how well a sample of voters (those voters who took the poll) reflects the general population (all voters), assuming that their voices are chosen randomly within the population you want to know about. It tells us how confident we can be that a poll is telling us how the whole population feels.
If people are selected randomly and some basic statistical assumptions are made, there is a mathematical formula for the margin of error. The way to understand a poll result such as “49 percent of American voters prefer McCain to Obama, with a margin of error of 3 percent” is that 49 percent of those questioned prefer McCain to Obama. What does the 3 percent margin of error mean? It means we can be 95 percent confident that the true value of support is somewhere between 46 (49 minus 3) to 52 (49 plus 3). The true value is how the whole population would vote. With a poll showing 49 percent support for McCain, and a margin of error of three points, we cannot conclude with confidence that Obama is ahead. (In the final count, according to Wikipedia, the popular vote was 52.9 percent in favor of Obama and 45.7 percent in favor of McCain.)
The idea of the 95 percent confidence interval is that it contains the true value around 95 percent of the time. For example, suppose that the true level of support for McCain is 47 percent. Then we should expect 19 out of 20 repeated polls (with new samples of people of course) to have margin of errors that contain 47 percent.
The margin of error can be made smaller by polling more people. A poll involving only a few hundred people may have a large margin of error, and a poll with thousands of people will have a small margin of error. Political polls by professional polling agencies are designed to poll just enough people to get a small margin of error (usually 2 or 3 percent). Typically, it takes just over 1,000 people to get a margin of error of 3 percent. To poll more people would be expensive, and to poll fewer people would mean less accurate results. Generally, the margin of error is proportional to the inverse of the square root of the number of people taking it. This means that to decrease the margin of error a little bit, you have to increase the number of people polled by a lot.
Why is this so important? The results of a poll may tell one story without the margin of error, and another with it. A small lead can turn into no lead at if the margin of error is too large.
Of course, all of this assumes that the poll talked to a sample of people representative of the whole population, a subject of some controversy. Many polls do not take representative samples, or they have a very low response rate, or they ask questions in a biased way, in which case a margin of error can be very misleading. As mentioned at the beginning, margin of error only refers to the effects of choosing a random sample. It does not provide any information about errors that come from other sources.