Basic Properties of Confidence Intervals

Posted by Beetle B. on Sun 16 July 2017

\(\newcommand{\Cov}{\mathrm{Cov}}\) \(\newcommand{\Corr}{\mathrm{Corr}}\) \(\newcommand{\Sample}{X_{1},\dots,X_{n}}\)

Assume you have a normal distribution with unknown \(\mu\) but known \(\sigma\) (highly implausible). A sample is collected. The 95% confidence interval for \(\mu\) is \(\left(\bar{x}-1.96\frac{\sigma}{\sqrt{n}},\bar{x}+1.96\frac{\sigma}{\sqrt{n}}\right)\)

This is the classical confidence interval (CI) (i.e. the frequentist).

The \(100(1-\alpha)\) % CI for the normal is \(\left(\bar{x}-z_{\alpha/2}\frac{\sigma}{\sqrt{n}},\bar{x}+z_{\alpha/2}\frac{\sigma}{\sqrt{n}}\right)\)

Deriving a Confidence Interval

Let \(\Sample\) be the sample, and \(\theta\) the quantity to be estimated. If you can find a random variable \(Y\) that satisfies:

  1. The random variable depends on \(\Sample\) and \(\theta\).
  2. The pdf of the random variable does not depend on \(\theta\) or any unknown parameters

Then you can find \(a,b\) such that \(P(a<Y<b)=1-\alpha\) (for any \(\alpha\)). Manipulate the expression to get \(P(A<\theta<B)=1-\alpha\). Then the CI for \(100(1-\alpha)\) is \((A,B)\).


Use \(Z=\frac{X-\mu}{\sigma/\sqrt{n}}\)


Use \(Y=2\lambda\sum X_{i}\) (you get the chi-squared distribution).

Bootstrap Confidence Intervals

You can use bootstrapping to estimate the confidence interval.

As an example, let \(B=1000\). Calculate \(\hat{\theta_{i}^{\ast}}\) for all of them, as well as the mean of all of them. Note the differences \(\hat{\theta^{\ast}}-\hat{\theta_{i}^{\ast}}\). Then look at the 25th largest and smallest values and you have your interval.