you do get any tied pairs of scores, subtract the number of tied scores
from the total number of paired scores to obtain your value of N.
7. The next step is to look up Table A2 in Appendix A to find out if the
result obtained from this data is statistically significant. As we did not
make a prediction about which version of the PDA application that
participants would prefer, we are only interested in significance levels
for two-tailed tests.
8. When assessing the significance (or not) of your obtained value of W,
remember that it is classed as being significant if it is equal to, or less
than the stated value in the table. From looking at Table A2, it shows
that the result for this example is significant (p 0.02).
9. How can this be interpreted? Well the result suggests that people have
a more positive attitude towards the keypad entry version of the PDA
application than they have for the stylus input version.
10. If you were writing this result up formally you would write something
like the following: Participants had a significantly more positive attitude
towards the keypad entry version of the PDA application than the
stylus input version, W 3, p 0.02).
Mann-Whitney U-test
This test should be used if you have a two condition unrelated design,
using the different participants to participate in both conditions. This is
154 Understanding Mobile Human–Computer Interaction
Participant Stylus Keypad Difference (d) Rank of d Rank of Rank of
12538 13 8 ()6
22439 15 10 ()8
33237 55()3
42936 76()4
5 38 34 4 3.5 ()2
6 31 30 1 1.5 ()1
72731 4 3.5 ()2
83940 1 1.5 ()1
92135 14 9 ()7
10 22 33 11 7 ()5
Total 3 39
Table 8.8 Initial Data Table for Calculating a Wilcoxon Signed-ranks Test
H6352-Ch08.qxd 7/18/05 3:40 PM Page 154
Data analysis 155
the non-parametric equivalent of the unrelated t-test. The example we
will use here will be similar to the example we used for the Wilcoxon
test outlined above. However, this time, the data will come from two
different groups of participants who used the two different versions of
the PDA device. The first condition required participants to use a stylus
to interact with the application and the second condition required the
second group of participants to use keypad entry. After using each ver-
sion of the application participants were asked to rate the usability of
the device using a Likert-type questionnaire. Let’s look at Technical Tip 4
to work through how to calculate this statistic.
Technical Tip 4
Calculating a Mann-Whitney U-test
The data obtained from this exercise came in the form of total scores for
each participant on the usability scale. The scale was based on a 7-point
scale. Once again, the higher the overall usability score, the more posi-
tively the participant rated the PDA application. The initial data table may
look like Table 8.9.
1. The formula for working out the Mann-Whitney U statistic is as follows:
UNN
NN
T
xx
x
12
()
2
1
Stylus Input Attitude Score Keypad Entry Attitude Score
24 38
24 39
32 37
29 36
38 34
31 30
27 31
31 40
21 35
31 33
Table 8.9 Participant Attitude Scores
H6352-Ch08.qxd 7/18/05 3:40 PM Page 155
2. The expressions used in the formula can be defined as follows:
N
1
the total number of participants in group 1
N
2
the total number of participants in group 2
N
x
total number of participants in the group with the largest rank total
T
x
the largest rank total score
3. In order to get data to put in these expressions, you will have to carry
out some calculations on your original data table. When it comes to
ranking the scores, you should rank the scores for both groups as a
single series of ranks. Once you have done this, your data should look
like Table 8.10.
4. Your formula should now look something like this:
5. According to my calculations U 512.
6. The next step is to look up Table A3 in Appendix A to find out if the
result obtained from this data is statistically significant. As we did not
make a prediction about which version of the PDA application partici-
pants would prefer, we are only interested in significance levels for
two-tailed tests.
U
10 10
10 11
2
143
156 Understanding Mobile Human–Computer Interaction
Stylus Input Rank (1) Keypad Entry Rank (2)
Attitude Score Attitude score
(group 1) (group 2)
24 2.5 38 17.5
24 2.5 39 19
32 10.5 37 16
29 5 36 15
38 17.5 34 13
31 8 30 6
27 4 32 10.5
31 8 40 20
21 1 35 14
31 8 33 12
Total 288 68 353 143
Table 8.10 Data Table for Carrying out Mann-Whitney U Calculation
H6352-Ch08.qxd 7/18/05 3:40 PM Page 156
Get Understanding Mobile Human-Computer Interaction now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
