Model / stats · All operations

Compare two means

A variant of Run a hypothesis test

Test whether two means differ, or whether one mean equals a given value, by a t-test.

Status

In arcform

Not built yet

arcform does not do this yet. Its place on the roadmap

What other tools call it

Meridian's classification; an independent blind rating agreed on 95.7% of 324 rows (variant, version 3).

Alteryx Designer (1)
  • Test of Meansread 2026-09-27

    Welch two-sample t-test of a numeric response between control and treatment groups

KNIME Analytics Platform (3)
  • Independent groups t-testread 2026-09-27

    Test whether the means of a column differ between two independent groups

  • Paired t-testread 2026-09-27

    Test whether the mean difference between two paired columns is zero

  • Single sample t-testread 2026-09-27

    Test whether a column's mean equals a given value

RapidMiner Studio (1)
  • T-Testread 2026-09-27

    Pairwise t-test of whether the mean performances of models differ significantly

Background

The most common use of a t-test is to test whether the means of two populations are significantly different.[1] A one-sample t-test tests whether the mean of a population is equal to a specified value.[1] A paired t-test compares matched pairs of similar units, or one group of units that has been tested twice.[1]

When to use it

The independent samples t-test is used when two separate sets of independent and identically distributed samples are obtained, such as subjects randomly assigned to a treatment group and a control group.[1] Paired t-tests have greater power, the probability of avoiding a false negative, than unpaired tests when the paired units are similar with respect to noise factors independent of membership in the two groups.[1] Choose the paired form only when each pair of values really belongs to one unit, or to a pair matched on purpose; rows that merely sit side by side are two independent samples. (Meridian's view)

Strictly, the name Student's t-test belongs to the two-sample form that assumes the two populations have equal variances, and the form that drops that assumption is sometimes called Welch's t-test.[1] Pre-testing for equal variances and then choosing between Student's t-test and Welch's t-test is not recommended.[2] Welch's t-test can be applied directly and without any substantial disadvantages compared with Student's t-test.[2] If the data are substantially non-normal and the sample size is small, the t-test can give misleading results.[1] The t-test is not robust in the presence of an outlier.[1] Look at both samples before reading the result, and treat a small sample with a long tail or a lone extreme value as a reason for caution. (Meridian's view)

The method

The one-sample t statistic divides the difference between the sample mean and the hypothesised mean by the sample standard deviation over the square root of the sample size n, and the test uses n − 1 degrees of freedom.[1] The paired t statistic is calculated from the average and standard deviation of the differences between all pairs, with n − 1 degrees of freedom where n is the number of pairs.[1] Unlike Student's t-test, Welch's t-test does not base the denominator of its statistic on a pooled variance estimate.[2] Welch's t-test approximates its degrees of freedom with the Welch–Satterthwaite equation.[2] The approximate degrees of freedom of Welch's t-test are real numbers, used as such in statistics-oriented software but rounded down to the nearest integer in spreadsheets.[2] In the stats_duck extension for DuckDB, the ttest_2samp function runs Welch's test by default and Student's pooled test when equal_var is set to true.[3]

William Sealy Gosset first published the t-distribution in English in 1908 in the journal Biometrika, using the pseudonym Student.[1] Gosset worked at the Guinness Brewery in Dublin, Ireland, and was interested in the problems of small samples.[1] Gosset devised the t-test as an economical way to monitor the quality of stout.[1] Ronald Fisher's work made the distribution well known as Student's distribution and the test as Student's t-test.[1] Welch's t-test is named for its creator, Bernard Lewis Welch, and is an adaptation of Student's t-test.[2]

Compared with the alternatives

In many cases a Z-test gives very similar results to a t-test, because the t-test converges to the Z-test as the size of the dataset increases.[1] The power of Welch's t-test comes close to that of Student's t-test, even when the population variances are equal and the sample sizes are balanced.[2]

Non-parametric methods such as the Mann–Whitney U test typically do not test for a difference of means, so should be used carefully if a difference of means is of primary interest.[1] The non-parametric counterpart to the paired samples t-test is the Wilcoxon signed-rank test for paired samples.[1]

Sources

  1. [1]Student's t-test - Wikipediasecondary source
  2. [2]Welch's t-test - Wikipediasecondary source
  3. [3]stats_duck – DuckDB Community Extensionsdocumentation