Paired t-test

1 Continuous Dependent Variable with normal distribution

1 (2 Level) Categorical Independent Variable

	Task Completion time
Subject	Interface 1	Interface 2
1	12.9	16
2	5.7	7.5
3	16	16
4	14.3	15.7
5	2.4	13.2
…

A paired t-test measures whether means from a within-subjects test group vary over 2 test conditions. The paired t-test is commonly used to compare a sample group’s scores before and after an intervention. In HCI practice, the paired t-test is also commonly
used to compare how a group of subjects perform in two different test conditions [12].

In ‘Tradeoffs in Displaying Peripheral Information’ [7], Maglio and Campbell test distraction level of peripheral displays. As part of the experiment,
test subjects perform a complex editing task. The performance measure is the number of correct edits they complete. In a second condition, subjects also periodically monitor a peripheral display containing miscellaneous news headlines. The performance measure
in the second condition incorporates the number of news headlines they remember.

The research team tests whether there is a significant difference in the number of edits the subjects’ complete in condition 1 (no peripheral display present) and condition 2 (peripheral display present). They demonstrate a valid application of the paired
t-test in their analysis. First, the paired t-test is applicable when measuring how a static group of subjects perform in two conditions, and this requirement is met. Second, the paired t-test is appropriate when the independent variable is dichotomous. In
their experiment, the two test conditions, (presence of a peripheral display or lack thereof) fulfill the requirement. Score on the editing task serves as the continuous dependent variable. Finally, 29 subjects participate in the experiment, so the research
team is marginally safe in assuming the dependent variable followed a normal distribution (the central limit theorem proves distribution is normal with a sample size of 30 or more). The research team finds a significant difference in the number of edits completed
in the two test conditions.

The paired t-test is similar to the repeated measures ANOVA test. Both can be used to compare how a static group performs in varying test conditions. The difference is that the paired t-test is used when the independent variable has two levels. Repeated
measures ANOVA is used when the independent variable has more than two levels. So, for example, if the researchers had tested three conditions, the appropriate test would have been repeated measures ANOVA rather than a paired t-test.

Values to report:

·        t value

·        degrees of freedom

·        p value