The margin of error and sample size of a poll matter a lot: here’s why

In virtually any discussion of a poll about the very close presidential race between Vice President Kamala Harris and former President Donald Trump, you will hear the phrase “within the poll’s margin of error.” These words indicate that it is a tight race with no clear leadereven if one of them has a slightly higher support rate, such as 48% to 47%.

The Oregonian/OregonLive Local Public Poll in October 2024.

Like the director of the Quinnipiac University Pollthat has taken the public’s pulse on policy issues and elections for the past thirty years, I’ve noticed that people have been paying more attention to this technical term since at least 2016.

For example, that year some polls in Florida indicated that Hillary Clinton was ahead by just a few percentage points from Trump. Journalists and the public largely—and wrongly—understood that apparent popular vote led to meaning Clinton would probably win.

But those 1 or 2 percentage points were within the margins of error of their polls. And Clinton lost Florida. In a poll about a political race, the margin of error tells readers what the likely outcome of an election will be.

What is a margin of error?

A poll consists of one or more questions asked to a small group of people and used to gauge the opinions of a larger group of people. The margin of error is a mathematical calculation of how accurate the survey results are – of how closely the small group’s answers match the larger group’s views.

If everyone in the larger group were surveyed, there would be no margin for error. But it is complicated, difficult and expensive to connect with so many people. The U.S. Census Bureau spent money $13.7 billion over several years in its most recent effort to count every person in the United States every ten years, and that is still the case was not able to include exactly everyone.

Pollsters don’t have as much time (or money), so they use smaller samples of the population. They try to identify themselves representative samples in which all members of the larger group have a chance to be included in the poll.

The group size is important

The calculation of how closely the poll reflects the views of the larger population is based on the size of the group being surveyed.

For example, a sample of 600 voters will have a larger margin of error – about 4 percentage points – than a sample of 1,000 voters, which has a margin of error of just over 3 percentage points.

The way the sample is chosen is also important: in 1936 the magazine Literary Digest polled people about the presidential election by sending surveys to phone owners, car owners and country club members. Everyone in this group was relatively wealthy, so they were not representative of the entire U.S. voting population. Calculating a margin of error would have been meaningless because the sample did not include all segments of the population.

A concrete example

Let’s use an example of how to understand the margin of error. If a poll shows that 47% of the group surveyed support Candidate A, and the margin of error is plus or minus 3 percentage points, this means that the percentage of the population that supports Candidate A is likely to be between 44% (47 minus 3 percentage points). lie. ) and 50% (47 plus 3).

A quick note: most polls report margins of error alongside another technical term: “confidence interval.” In the most rigorous reporting of polls, you might see a sentence at the end that says something like: “The margin of error is plus or minus 3 percentage points, at a 95% confidence interval.” What that all means is this: imagine if 100 different random samples of the same size were selected from the larger group and then asked the same questions in the poll. The 95% confidence interval means that 95% of the time the answers from those other polls are within 3 percentage points of the answers reported in this one poll.

Compare support between candidates

The concept of margin of error becomes more complex when we look at the differences in support between two candidates. If the margin of error is plus or minus 3 percentage points, the margin of error on the difference between them is about double that – or six percentage points, in this example.

That’s because the margin of error here is a combined margin of error, and refers not only to the vote percentage for candidate A, but also to the vote percentage for the other candidate.

To look back on 2016, the final Quinnipiac University Poll in Florida before Election Day showed Clinton with 46% support and Trump with 45% support. The margin of error was 3.9 percentage points, meaning Clinton would likely receive between 42.1% and 49.9% of the vote, and Trump would likely receive between 41.1% and 48.9% of the vote.

The actual result was that Trump won Florida with 48.6%compared to Clinton’s 47.4%. These results were within our poll’s margin of error, meaning we were right to declare it “too close to call” — and we would have been wrong to say Clinton was ahead.

2024 will be exciting elections

In the current election cycle, there have been many media reports about polls not including information about the margin of error.

Omitting that information, or downplaying its significance, can help the media provide a quick, simple picture of the state of the race. Technology can seem precise in the modern age of the internet and artificial intelligence.

But polls aren’t that accurate. It’s an inexact science. It is the job of pollsters to capture snapshots of the complexity of human nature at a given moment. People’s thoughts can change and new information can emerge as the campaigns unfold.

With the presidential election in its final weeks, our polls are finding a quite tight and stable racewith most voters telling us they have made up their minds. Because the difference between the presidential candidates in swing states is within the margin of error, the fall 2024 election polls are telling Americans to hold their breath and make sure they vote because it will likely be a squeaker.