Student's t-Test

The p-value is the probability of observing a test statistic at least as extreme as yours, assuming H₀ is true. It is NOT the probability that H₀ is true. A p-value of 0.03 means: if the null were true, you'd see a result this extreme only 3% of the time by chance — unusual enough to reject. But 5% significance means 1-in-20 tests will reject a true null purely by accident (Type I error). Smaller α is stricter but increases Type II errors (missing real effects). The chart shows this directly: if the red line is past the amber threshold, p < α.

At heart, the t-statistic is just a signal-to-noise ratio: the gap between your group means (the signal) divided by the standard error (the noise from random sampling). A big t means the difference stands out clearly from the scatter; a small t could easily be luck. The practical lesson: the same difference in means becomes more convincing as your samples grow or their spread shrinks, because that shrinks the noise in the denominator.

You might expect degrees of freedom to be a tidy integer — a count of independent pieces of information. But the Welch t-test uses the Welch–Satterthwaite formula, which typically returns a fractional value like df = 17.4. It is a deliberate approximation: when two groups have unequal variances, no exact whole-number df exists, so the method interpolates to match the true distribution as closely as possible. That is why Welch's df usually differs from the classic n₁ + n₂ − 2.