A point estimate is a single dot with no sense of its own error. A confidence interval fixes that: instead of one guess, it reports a range of plausible values together with a confidence level such as 95\%.

We build it from the estimate and its standard error — roughly, the estimate plus-or-minus a few standard errors. The empirical rule is why "a few": about 95\% of the sampling distribution sits within about two standard errors of the centre.

What the 95% really describes

Here is the subtle part. A 95\% confidence interval is built by a procedure with this property: if you repeated the whole experiment many times — new sample, new interval each time — then about 95\% of those intervals would capture the true \mu. The confidence lives in the recipe, not in any one interval it produces.

So it is tempting but wrong to say "there is a 95\% chance \mu is in this interval". Once computed, your interval either contains the fixed \mu or it does not — there is no probability left in it. The 95\% is the long-run hit rate of the method.

A stack of intervals, one true μ

The vertical line is the fixed true \mu. Each horizontal bar is one confidence interval from one sample, centred on that sample's estimate. Most bars cross the line — they captured \mu. The odd one out (drawn in a warning colour) sits entirely to one side: it missed. Over many intervals, the fraction that catch \mu hovers near the confidence level.

Asking for more confidence means missing less often — and the only way to miss less often with the same data is to make every interval wider. Confidence and width pull in the same direction.