Colorado State University — Pueblo; Fall 2016
Math 156, Introduction to Statistics
Confidence Intervals for the Population Mean

The Context

Suppose we have some population about which we are interested in the values of a particular numerical measurement. If we pick individuals from the population at random, then we can think of the values of this measurement as a quantitative random variable $X$. We might want to know

but, if the population size $N$ is large, it may be impractical simply to go out and collect all the data to compute $\mu$ and $\sigma$.

Instead, we might take a sample of size $n$ and compute

If $n$ is fairly large and if we think the sample is well (very randomly!) chosen, we might guess that $\mu\approx\overline{X}$.

The point of a Confidence Interval is to make a guess for how close $\mu$ actually is to $\overline{X}$, and in fact to be precise enough to say what is the probability that $\mu$ is a certain distance from $\overline{X}$. That is: if we choose a confidence level $L$ (which is a number from $0$ to $1$, often expressed as a percentage), we want to make an interval of real numbers centered at $\overline{X}$ with the probability of $\mu$ being in that interval equalling (at least) $L$.

Known Population Standard Deviation $\sigma$

The Central Limit Theorem tells us that the distribution of $\overline{X}$ as we take repeated simple random samples is approximately normal with mean $\mu$ and standard deviation $\sigma/\sqrt{n}$, if $n$ is large. By the Empirical Rule, then:

  1. 68% of the time we take samples, $\overline{X}$ will be within $\sigma/\sqrt{n}$ of $\mu$. Equivalently, 68% of the time we take samples, $\mu$ will be within $\sigma/\sqrt{n}$ of $\overline{X}$. In other words, 68% of the time we take samples, $\mu$ will happen to lie in the interval from $\overline{X}-\sigma/\sqrt{n}$ to $\overline{X}+\sigma/\sqrt{n}$.
  2. Likewise, 95% of the time we take samples, $\overline{X}$ will be within $2\sigma/\sqrt{n}$ of $\mu$. Equivalently, 95% of the time we take samples, $\mu$ will be within $2\sigma/\sqrt{n}$ of $\overline{X}$. In other words, 95% of the time we take samples, $\mu$ will happen to lie in the interval from $\overline{X}-2\sigma/\sqrt{n}$ to $\overline{X}+2\sigma/\sqrt{n}$.
  3. Likewise, 99.7% of the time we take samples, $\overline{X}$ will be within $3\sigma/\sqrt{n}$ of $\mu$. Equivalently, 99.7% of the time we take samples, $\mu$ will be within $3\sigma/\sqrt{n}$ of $\overline{X}$. In other words, 99.7% of the time we take samples, $\mu$ will happen to lie in the interval from $\overline{X}-3\sigma/\sqrt{n}$ to $\overline{X}+3\sigma/\sqrt{n}$.

Let us extract a general formula from this:

The confidence interval at confidence level $L$ for the population mean $\mu$ of a normally distributed population with known population standard deviation $\sigma$ has endpoints $\overline{X}\pm z^*_L\sigma/\sqrt{n}$, if the sample size $n$ is fairly large (typically, $n\ge30$), where $\overline{X}$ is the sample mean.

Here $z^*_L$ is the critical value for the Normal distribution with probability $L$, meaning that the standard Normal distribution has probabilty $L$ for $-z^*_L<Z<z^*_L$. A table of these values is useful:

 confidence level $L$  .5 .8 .9 .95 .99 .999
critical value $z^*_L$   .674   1.282   1.645   1.960   2.576   3.291 
It is important to keep in mind that the $\mu$ is not changing. Rather, the sample data and therefore the sample average $\overline{X}$ changes every time we choose a new (random) sample. And the confidence level $L$ means that with probability $L$, each interval we have computed will contain the value $\mu$ (which is fixed, but unkown). In other words, a fraction of size $L$ of the samples we would take, if we had the resources to keep taking, again and again, new SRSs, would give us new $\overline{X}$s and therefore new confidence intervals which would contain $\mu$. So it is the method in which we have confidence, not any particular interval we compute. The method produces intervals which are "correct," in the sense that they contain the true (unknown) value of $\mu$, with probability $L$.

Since that term $z^*_L\sigma/\sqrt{n}$ tells us how much error there could be in the estimate of $\overline{X}$ for $\mu$, we call it the margin of error.

Unknown Population Standard Deviation $\sigma$

Pretty much the same thing works even if we don't know the population standard deviation $\sigma$. Instead, we use the next best thing: the sample standard deviation $s$. However, we have to take our critical values from a different distribution, called Student's $t$-distribution. The $t$-distribution has "fatter tails" to compensate for the fact that $s$ is only an approximation for the value $\sigma$, which we would have preferred to use.

In fact, the bigger the sample size $n$, the closer the sample standard deviation $s$ will be to the population standard deviation $\sigma$, so the closer can be that version of the $t$-distribution to the standard Normal distribution. For smaller values of $n$, the corresponding $t$-distribution will instead have very fat tails. So actually a table of critical values of the $t$-distribution will have to have different values depending upon $n$.

To make matters slightly more complicated, we don't just use the sample size $n$ to pick which row of the table to use: in fact, we use instead the degrees of freedom $df$, which fortunately is just $df=n-1$. Here is a table:
   confidence level $L$ 
$df$ .5 .8 .9 .95 .99 .999
  1    1.000   3.078   6.314   12.71   63.66   636.62 
  2    0.816   1.886   2.920   4.303   9.925   31.599 
  3    0.765   1.638   2.353   3.182   5.841   12.924 
  4    0.741   1.533   2.132   2.776   4.604   8.610 
  5    0.727   1.476   2.015   2.571   4.032   6.869 
  6    0.718   1.440   1.943   2.447   3.707   5.959 
  7    0.711   1.415   1.895   2.365   3.499   5.408 
  8    0.706   1.397   1.860   2.306   3.355   5.041 
  9    0.703   1.383   1.833   2.262   3.250   4.781 
  10    0.700   1.372   1.812   2.228   3.169   4.587 
  11    0.697   1.363   1.796   2.201   3.106   4.437 
  12    0.695   1.356   1.782   2.179   3.055   4.318 
  13    0.694   1.350   1.771   2.160   3.012   4.221 
  14    0.692   1.345   1.761   2.145   2.977   4.140 
  15    0.691   1.341   1.753   2.131   2.947   4.073 
  16    0.690   1.337   1.746   2.120   2.921   4.015 
  17    0.689   1.333   1.740   2.110   2.898   3.965 
  18    0.688   1.330   1.734   2.101   2.878   3.922 
  19    0.688   1.328   1.729   2.093   2.861   3.883 
  20    0.687   1.325   1.725   2.086   2.845   3.850 
  21    0.686   1.323   1.721   2.080   2.831   3.819 
  22    0.686   1.321   1.717   2.074   2.819   3.792 
  23    0.685   1.319   1.714   2.069   2.807   3.768 
  24    0.685   1.318   1.711   2.064   2.797   3.745 
  25    0.684   1.316   1.708   2.060   2.787   3.725 
  26    0.684   1.315   1.706   2.056   2.779   3.707 
  27    0.684   1.314   1.703   2.052   2.771   3.690 
  28    0.683   1.313   1.701   2.048   2.763   3.674 
  29    0.683   1.311   1.699   2.045   2.756   3.659 
  30    0.683   1.310   1.697   2.042   2.750   3.646 
  40    0.681   1.303   1.684   2.021   2.704   3.551 
  60    0.679   1.296   1.671   2.000   2.660   3.460 
  80    0.678   1.292   1.664   1.990   2.639   3.416 
  100    0.677   1.290   1.660   1.984   2.626   3.390 
  1000    0.675   1.282   1.646   1.962   2.581   3.300 

Using the $t$-distribution critical value $t^*_{L,df}$ corresponding to the desired confidence level $L$ and to the degrees of freedom $df$ depending upon the sample size, from the above table, we have
The confidence interval at confidence level $L$ for the population mean $\mu$ of a normally distributed population with unknown population standard deviation has endpoints $\overline{X}\pm t^*_{L,df}s/\sqrt{n}$, where $n$ is the sample size, $\overline{X}$ is the sample mean, and $s$ is the sample standard deviation.