11. The next step is to calculate the degrees of freedom. This is calculated
as follows:
df N
A
N
B
2. In our case this is 20 2. Therefore, the df 18.
12. Now you are ready to see if your results are statistically significant. In
order to this, consult Table A1 in Appendix A. As we did not make any
predictions about which group would find the information on train
times quicker than the other, we are interested in the critical t values
for a two-tailed test. If we had made a prediction that females would
find the train departure information quicker than males, we would be
looking at columns related to one-tailed values.
13. Once you have your tails sorted out, look for the column where it says
df 18. If your value of t is larger than the figure listed in that column
we can say that you have found a significant difference in search times
between females and males at the 5 per cent level (p 0.05). If it is
lower, we did not find a significant difference in search times between
males and females in your study. The final thing to mention here is how
to report this finding in your written report. Conventionally you would
report this finding as follows: t 0.89, df 18, two-tailed ns (where
ns stands for not significant). This could be written out more formally
as the mean time taken for females to find train departure information
(M 225.1 secs) using the PDA application is not significantly quicker
than the average time taken by male users (M 210.1 secs), t 0.89,
df 18, two-tailed ns.
Wilcoxon signed-ranks test
This test should be used if you have a two condition related design, using
the same participants (or matched participants) to participate in both con-
ditions. The aim of this test is to compare participants’ performance in the
two conditions to find out whether or not there is any difference between
the scores obtained. This test is the non-parametric equivalent of the
related t-test. The example we will use here will be data from a group of
participants who used two different methods of interaction to carry out a
web search using a PDA device. The first condition required participants
to use a stylus to interact with the application and the second condition
required them to use keypad entry. After using each version of the appli-
cation participants were asked to rate the usability of the device using a
Likert-type questionnaire. Let’s look at Technical Tip 3 to work through
how to calculate this statistic.
152 Understanding Mobile Human–Computer Interaction
H6352-Ch08.qxd 7/18/05 3:40 PM Page 152
Data analysis 153
Technical Tip 3
Calculating a Wilcoxon singed-ranks test
The data obtained from this study comes from the attitude score obtained
from each participant on the Likert usability scale described above. The
scale was based on a 7-point scale. The higher the overall usability score,
the more positively the participant rated the PDA application. Table 8.7
provides information on what the initial data table may look like:
1. The first step in working out this formula is to calculate the difference
between each pair of scores, assigning plus or minus signs where
appropriate.
2. The next step is to rank the differences between the scores obtained
for the two conditions (ignoring the plus or minus signs this time).
3. After that, rank the plus difference and rank the minus differences.
4. Your data table should now look something like Table 8.8.
5. The next step is to take the smaller of the totals obtained for the plus
and minus rank differences. This gives us the value of a statistic called
W (remember this is called the Wilcoxon test) and this will be used to
look up the appropriate table for statistical significance.
6. The next step is to calculate the number of pairs of participants that do
not have tied scores. From the example above we can see that there
are no tied pairs of scores, therefore in this case N 10. However, if
Participant Stylus Input Attitude Score Keypad Entry Attitude Score
125 38
224 39
332 37
429 36
538 34
631 30
727 31
839 40
921 35
10 22 33
Table 8.7 Participant Attitude Scores for Stylus Input and Keypad Entry
H6352-Ch08.qxd 7/18/05 3:40 PM Page 153

Get Understanding Mobile Human-Computer Interaction now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.