Imagine that you are trying to find out how many Canadians have taken at least two weeks of vacation in the past year. You could ask every Canadian about his or her vacation schedule to get the answer, but this would be expensive and time consuming.
To save time and money, you would probably survey a smaller group of Canadians. However, your finding may be different from the actual value if you had surveyed the whole population. That is, it would be an estimate. Each time you repeat the survey, you would likely get slightly different results.
Commonly, when researchers present this type of estimate, they will put a confidence interval (CI) around it. The CI is a range of values, above and below a finding, in which the actual value is likely to fall. The confidence interval represents the accuracy or precision of an estimate.
How confidence intervals are used
We often see CIs in newspapers when the results of polls are released. An example from the Globe and Mail newspaper regarding the mayoral race in Toronto read, "52 per cent [of survey respondents] said they would have voted for Mr. Miller if the election had been held last week. The margin of error is plus or minus 4.4 percentage points, 19 times out of 20."
The "margin of error" represents the confidence interval. It is the range from 47.6 to 56.4 per cent - that is, 52 per cent plus or minus 4.4 percentage points. The researchers are confident that if other surveys had been done, then 95 per cent of the time — or 19 times out of 20 — the findings would fall in this range.
The 95 per cent confidence level is used most often in research; it is a generally accepted standard. However, researchers can calculate CIs at any level of significance, such as 90 per cent or 99 per cent. The significance level simply indicates how precise they are willing to be.
What factors influence a confidence interval?
A narrow or small confidence interval indicates that if we were to ask the same question of a different sample, we are reasonably sure we would get a similar result. A wide confidence interval indicates that we are less sure and perhaps information needs to be collected from a larger number of people to increase our confidence.
Confidence intervals are influenced by the number of people that are being surveyed. Typically, larger surveys will produce estimates with smaller confidence intervals compared to smaller surveys. Other factors will include the accuracy of the measurements in a survey. If measurements are less accurate, it will likely increase confidence intervals.
Why are confidence intervals important?
Because confidence intervals represent the range of scores that are likely if we were to repeat the survey, they are important to consider when generalizing results. In the example with Mr. Miller, how confident would you be in saying that more than half of Torontonians would vote for Miller?
If you repeated the survey again, you may get a value of 47.6 per cent, which lies within your 95 per cent CI. Therefore, you may not be comfortable with such a statement. On the other hand, you would likely be more confident saying that at least 45 per cent of voters will cast their vote for Miller.
Source: At Work, Issue 47, Winter 2007: Institute for Work & Health, Toronto