Statistical Test Chooser

Which statistical test fits your data? A step-by-step guide in plain language to 39 tests and models, each with assumptions, effect size, SPSS path and verified R and Python code with plots

Also in the Workspace · runs in your browser

Why use it

Choosing the wrong test is among the most frequent statistical errors in medical papers: paired data analysed as independent, many t tests instead of an ANOVA, chi-square with tiny expected counts, Pearson correlation on skewed data, no effect size. Most test selectors stop at a name. This one also explains why, shows how to run the analysis in the three tools used in teaching and research (SPSS, R, Python), and how to read and report it.

About

The Statistical Test Chooser guides a researcher from the research question to the analysis. It asks one question at a time — what you want to find out, the type of the outcome, how many groups, whether they are independent or paired, and, when it matters, the shape of the data — and only the questions relevant to the path taken. Every option is explained in plain words with a clinical example, so that answering the questions is also a way of getting to know the structure of one’s own data. The result is a recommended test with the alternatives to use if an assumption fails, and the checks to run first. Each of the 39 tests and models has its own page: what question it answers and the idea behind it; an example study with the data layout; the hypotheses; each assumption, how to check it and what to do if it is violated; the effect size with its formula and interpretation; post-hoc procedures; the SPSS menu path and what to read in the output; complete R and Python scripts with descriptive statistics, assumption checks, the test, the effect size and a plot; how to read the output; a reporting template; common pitfalls; references; and a link to the Sample Size Calculator for the same design.

When to use

Planning the statistical analysis section of a protocol or thesis
Before analysing data, to confirm the test matches the design and the variables
Learning or teaching biostatistics: each path is a worked lesson on data types and assumptions
Reviewing a manuscript: checking whether the test used fits the design
Getting runnable R or Python code to start an analysis

Method

The five questions (in order): 1. What do you want to find out? compare · relate · predict · agreement · diagnostic accuracy · time to event 2. What type is the outcome? measurement · ordinal · binary · nominal · count (· time to event) 3. How many groups / occasions? one vs reference · two · three or more · two factors 4. Independent or paired? 5. Shape of the data (only when it matters): normality · equal variances · small expected counts Example paths: measurement, 2 independent groups, normal → Welch / Student t test (else Mann–Whitney) measurement, 3+ paired occasions, normal → repeated-measures ANOVA (else Friedman) binary, 2 independent groups, large counts → χ² test of independence (small counts: Fisher) binary, 2 paired measurements → McNemar two measurements, linear, no outliers → Pearson r (else Spearman / Kendall)

Pearls & pitfalls

The design (independent or paired) and the outcome type decide most of the choice; normality matters less than is often taught, especially with 30+ per group.
Check normality within each group, of the paired differences, or of the regression residuals — never of the pooled data.
A non-significant normality or Levene test with small samples does not prove the assumption holds.
Welch’s t test is a safe default for two independent means; it loses almost nothing when variances are equal.
Compare three or more groups with one overall test followed by a post-hoc procedure, not with many pairwise tests.
Always report an effect size with a confidence interval, not only a p-value.
Decide the analysis before seeing the data, and register it in the protocol.

Next steps

Check the assumptions listed, run the analysis with the code provided (replacing the example data with yours), report the effect size with its confidence interval, and calculate the sample size for the chosen test with the Sample Size Calculator.

Evidence

The decision rules follow chapter Appendix D (“five questions” and test-choice tables) of the course manual Applied Biostatistics (G. de Macedo Oliveira) and the author’s teaching guides (tests, SPSS paths, effect sizes and post-hoc procedures), extended with standard methods not covered in the course: Cochran’s Q, two-proportion z test, Poisson and negative binomial regression, ordinal and multinomial regression, Kaplan–Meier/log-rank and Cox models, kappa, ICC and Bland–Altman. Each test page cites its methodological references. Every R and Python script was executed, and both languages give the same statistic and p-value for the example data (differences in library defaults, such as continuity corrections, are stated in the code). The decision tree is checked automatically: all 75 possible paths end in a documented test.

References

Altman DG. Practical Statistics for Medical Research. London: Chapman & Hall; 1991. — link
Field A. Discovering Statistics Using IBM SPSS Statistics. 5th ed. London: SAGE; 2018.
Marôco J. Análise Estatística com o SPSS Statistics. 8th ed. Pero Pinheiro: ReportNumber; 2021.
Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum; 1988. — link
Lang TA, Altman DG. Basic statistical reporting for articles published in biomedical journals: the SAMPL guidelines. In: Smart P, Maisonneuve H, Polderman A, eds. Science Editors’ Handbook. EASE; 2013. — link
Wasserstein RL, Lazar NA. The ASA statement on p-values: context, process, and purpose. Am Stat. 2016;70(2):129–133. — link
Strasak AM, Zaman Q, Pfeiffer KP, Göbel G, Ulmer H. Statistical errors in medical research — a review of common pitfalls. Swiss Med Wkly. 2007;137(3–4):44–49. — link
Delacre M, Lakens D, Leys C. Why psychologists should by default use Welch’s t-test instead of Student’s t-test. Int Rev Soc Psychol. 2017;30(1):92–101. — link