Hypothesis Testing, Error and Sample Size
Start with the single idea everything else hangs on: you can almost never measure the whole population, so you study a sample and reason backwards. A study measures a handful of women and asks what that handful implies about every comparable woman you will never meet. Statistics is the discipline of doing that reasoning honestly — quantifying how much the sample might mislead you, and how confident you may be in what it suggests.
Because you are reasoning from a sample, if you repeated the same study you would get a slightly different result. The point estimate would move; sometimes the conclusion would flip. Everything in this chapter — confidence intervals, p values, Type I and Type II error, power, sample size — is machinery for handling that unavoidable sampling variation. Hold this picture and the rest follows: it is not a truth machine, it is a way to keep sampling chance from fooling you.
Hypothesis testing in particular is a disciplined way to ask whether observed data are compatible with a specified "no effect" model. A p value does not prove a treatment works, a non-significant result does not prove equivalence, and a large sample does not rescue a biased study. The examiner expects you to interpret uncertainty clinically: what effect was sought, how precise is the estimate, what errors are possible, and was the study large enough to answer the question?
In O&G the consequences are immediate. A false-positive trial may introduce a useless or harmful intervention into antenatal care. A false-negative trial may discard a cheap treatment that prevents eclampsia or death due to bleeding. A poorly powered audit may label a service change as ineffective when it was simply unable to detect a meaningful change. A huge routine dataset may make a trivial difference statistically significant while leaving clinical practice unchanged.
Sample, Population and Sampling Variation
Two words do a lot of work, so define them precisely.
- The population is everyone the conclusion is meant to apply to — for example, all pregnant women at high risk of pre-eclampsia. It is usually too large or too nebulous to measure in full.
- The sample is the actual women in the study. Every number a study reports (a mean, a proportion, a risk ratio) is a point estimate calculated from the sample, used to guess the unmeasurable population value.
The honest investigator keeps two distinct ideas separate:
| Idea | Question it answers | Driven mainly by |
|---|---|---|
| Precision | If I repeated this study, how much would the estimate bounce around? | Sample size and variability (random error) |
| Accuracy (validity) | Does my sample fairly represent the population at all? | Sampling and design (bias) |
A bigger sample tightens precision — the estimate wobbles less from study to study — but size cannot cure bias. If women were selected or treatments allocated in a systematically unrepresentative way, the estimate is centred in the wrong place, and more women only pin down the wrong answer more confidently. A small clean randomised trial can be more trustworthy than a large skewed cohort. This is why a single huge routine dataset is not automatically the strongest evidence, and why the rest of this chapter assumes the design is sound before it worries about the numbers.
Describing the Data Before Testing It
Before any test, you must summarise what kind of data you have, because the data type dictates everything downstream — how you summarise it, how you display it, and which test is legitimate.
| Data type | Meaning | O&G example | Typical summary |
|---|---|---|---|
| Continuous (quantitative) | A measurement on a continuous scale | Birthweight, blood pressure, haemoglobin | Mean and SD, or median and IQR |
| Ordinal | Ordered categories with no fixed spacing | Mild / moderate / severe symptoms | Median and IQR, proportions |
| Categorical (nominal) | Unordered groups | Ethnicity, mode of delivery | Counts and proportions |
| Binary | Two categories | Pre-eclampsia yes/no, alive/dead | Proportion or risk |
| Time-to-event | Whether and when an event occurs, with differing follow-up | Time to delivery after PPROM | Rate, survival curve |
For a continuous variable, three summary numbers describe the centre. The mean is the arithmetic average. The median is the middle value when observations are ranked, and the mode is the commonest value. In a perfectly symmetrical (normal) distribution these coincide. When data are skewed — as theatre-to-delivery intervals or hospital length of stay almost always are, with a long right tail — the mean is dragged toward the tail while the median stays put. For skewed data the median and interquartile range describe the typical patient more honestly than the mean and standard deviation, which is why maternity audits quote median decision-to-delivery times, not means.
Variability: Standard Deviation Versus Standard Error
A centre alone is not enough; you must also describe spread, and here a distinction trips up most candidates.
- The standard deviation (SD) describes how spread out the individual measurements are. A large SD means women differ widely from one another. The SD does not shrink as you recruit more women — it is a property of the population, not of your effort.
- The standard error (SE) describes how much the sample estimate itself (for example, the sample mean) would bounce around if you repeated the whole study. It is the SD of that imaginary distribution of repeated results, and it gets smaller as the sample grows, because a larger sample pins the estimate down more tightly.
Put crudely: SD answers "how different are the women?"; SE answers "how reliable is my number?" Confidence intervals are built from the standard error, which is why intervals narrow with sample size while the underlying spread of women does not. Confusing the two is a classic exam error — and reporting a tight SE as if it described the patients, or a wide SD as if it described the estimate, misleads the reader.
The Normal Distribution and the 1.96 Anchor
Much of medical statistics rests on the normal (Gaussian) distribution — the symmetrical bell-shaped curve. Its usefulness comes from one fixed property: a known fraction of the area sits within a set number of standard deviations of the mean.
| Coverage | Standard deviations either side of the mean |
|---|---|
| ~68% | 1 |
| 95% | 1.96 (commonly rounded to 2) |
| 99% | 2.58 |
The number 1.96 is the single most reused constant in this whole field. A 95% confidence interval is, in the simplest case, the point estimate plus and minus 1.96 standard errors; the 1.96 also appears inside the sample-size formula as the term tied to a 5% false-positive rate. Recognising where it comes from demystifies both. The deeper reason the normal curve appears so often is that even when individual measurements are not normally distributed, the average of a reasonably large sample tends toward a normal distribution — which is why means, proportions and effect estimates can usually be handled with normal-based methods once samples are not tiny.
Binary and event data follow their own distributions (the binomial for yes/no counts, the Poisson for events accumulating over time), but for large samples these are well approximated by the normal distribution, so the same machinery — and the same 1.96 — carries across.
Choosing the Right Test
A statistical test compares groups or relationships, but each test makes assumptions, and using one whose assumptions are violated produces a misleading p value. The first fork is parametric versus non-parametric.
- Parametric tests assume the data follow a particular distribution, usually the normal. When that holds they are more powerful — they extract more evidence from the same number of patients.
- Non-parametric tests make no distributional assumption; they typically rank the data instead. They are robust to skew and outliers but less powerful, so they need more patients to show the same effect.
The simplest way to decide is to plot the data (a histogram) and look: roughly symmetrical and bell-shaped, with a reasonable sample size, supports a parametric test; clearly skewed, heavily outlying, or a very small sample favours a non-parametric one. The choice then depends on the data type and the comparison: