Sample size: the question to ask before any other

Why sample size comes first

Of all the statistical decisions made during the design of a clinical study, the sample size calculation is the one most often treated as a formality: a number requested by a protocol template, produced in a few minutes, and then not revisited. This is a mistake. The sample size determines what a study is able to say. A study that is too small will frequently fail to detect a real effect, not because the treatment does not work, but because the design was never able to observe it clearly enough. A study that is too large uses resources, and participant time, that a smaller, well justified design could have avoided.

In our experience, most problems with sample size are not calculation errors. They come from skipping the discussion that should happen before any formula is applied: what difference would actually matter, and how much uncertainty the study is prepared to accept.

The four numbers every calculation needs

Every standard sample size formula, regardless of the study design, is built from four inputs.

• The effect size. The smallest difference, or association, that would be considered meaningful if it were confirmed.
• The variability of the outcome. How much the measurement is expected to differ between participants, usually taken from a standard deviation, a variance, or an event rate reported in earlier studies.
• The significance level, alpha. The accepted probability of concluding there is an effect when in fact there is none. By convention, 0.05.
• The power. The probability of detecting the effect if it is truly present. By convention, 80% or 90%.

Of these four inputs, the effect size is the one most often estimated loosely, and it is also the one with the greatest influence on the final number. A modest change in the assumed effect size can double, or halve, the number of participants required.

What "clinically meaningful" actually means

A common error is to define the effect size from what the data are expected to show, rather than from what would matter to a patient or a clinical decision. These are not the same question. A trial can be powered to detect a difference of two points on a scale simply because that is what a pilot study observed, while two points may have no clinical relevance at all. The correct starting point is clinical: what is the smallest change that would justify a different treatment decision. Only once that number exists does it make sense to ask whether the study, at a feasible size, is able to detect it.

Allocation ratio, and a mistake that is easy to miss

When a study compares two or more groups, the way participants are allocated between them also affects the required size. Equal allocation, for instance 1:1, is generally the most statistically efficient choice. Unequal ratios are sometimes used for practical reasons, such as limited availability of an active comparator, but they increase the total sample size needed to preserve the same power. A separate, and surprisingly common, mistake happens in the final step of the calculation: rounding. A formula will rarely output a whole number, and rounding down, even by one or two participants per group, can meaningfully reduce the power that was just calculated. The correct approach is to round up, always, and to round after applying the allocation ratio, not before. It is a small detail, but we have seen it change a trial from adequately powered to underpowered without anyone noticing until the analysis stage.

Precision, not power, for descriptive studies

Not every study is designed around a hypothesis test. Observational and descriptive studies, including many registry-based and real-world evidence analyses, are often built to estimate a quantity, for example a prevalence or an incidence rate, with a specified precision, rather than to detect a difference against a comparator. In these cases, the relevant calculation is based on the width of the confidence interval the study is expected to produce, not on power. Applying a power-based formula to a purely descriptive study is a frequent, and avoidable, source of confusion between what a study is designed to test and what it is actually designed to estimate.

A number is not a guarantee

A sample size calculation is only as reliable as its assumptions. Recruitment is slower than planned, event rates are lower than the literature suggested, dropout is higher than expected. None of this means the initial calculation was wrong. It means that sample size planning should include a realistic allowance for attrition, and, where the design allows it, a plan for how the assumptions will be checked once the study is underway, for example through a blinded sample size re-estimation. A single number produced before the first participant is enrolled cannot account for everything that will happen during the study, but a well reasoned one gives the study its best chance of answering the question it was designed to ask.