you do get any tied pairs of scores, subtract the number of tied scores

from the total number of paired scores to obtain your value of N.

7. The next step is to look up Table A2 in Appendix A to find out if the

result obtained from this data is statistically significant. As we did not

make a prediction about which version of the PDA application that

participants would prefer, we are only interested in significance levels

for two-tailed tests.

8. When assessing the significance (or not) of your obtained value of W,

remember that it is classed as being significant if it is equal to, or less

than the stated value in the table. From looking at Table A2, it shows

that the result for this example is significant (p 0.02).

9. How can this be interpreted? Well the result suggests that people have

a more positive attitude towards the keypad entry version of the PDA

application than they have for the stylus input version.

10. If you were writing this result up formally you would write something

like the following: Participants had a significantly more positive attitude

towards the keypad entry version of the PDA application than the

stylus input version, W 3, p 0.02).

Mann-Whitney U-test

This test should be used if you have a two condition unrelated design,

using the different participants to participate in both conditions. This is

154 Understanding Mobile Human–Computer Interaction

Participant Stylus Keypad Difference (d) Rank of d Rank of Rank of

12538 13 8 ()6

22439 15 10 ()8

33237 55()3

42936 76()4

5 38 34 4 3.5 ()2

6 31 30 1 1.5 ()1

72731 4 3.5 ()2

83940 1 1.5 ()1

92135 14 9 ()7

10 22 33 11 7 ()5

Total 3 39

Table 8.8 Initial Data Table for Calculating a Wilcoxon Signed-ranks Test

H6352-Ch08.qxd 7/18/05 3:40 PM Page 154

Data analysis 155

the non-parametric equivalent of the unrelated t-test. The example we

will use here will be similar to the example we used for the Wilcoxon

test outlined above. However, this time, the data will come from two

different groups of participants who used the two different versions of

the PDA device. The first condition required participants to use a stylus

to interact with the application and the second condition required the

second group of participants to use keypad entry. After using each ver-

sion of the application participants were asked to rate the usability of

the device using a Likert-type questionnaire. Let’s look at Technical Tip 4

to work through how to calculate this statistic.

Technical Tip 4

Calculating a Mann-Whitney U-test

The data obtained from this exercise came in the form of total scores for

each participant on the usability scale. The scale was based on a 7-point

scale. Once again, the higher the overall usability score, the more posi-

tively the participant rated the PDA application. The initial data table may

look like Table 8.9.

1. The formula for working out the Mann-Whitney U statistic is as follows:

UNN

NN

T

xx

x

12

()

2

1

Stylus Input Attitude Score Keypad Entry Attitude Score

24 38

24 39

32 37

29 36

38 34

31 30

27 31

31 40

21 35

31 33

Table 8.9 Participant Attitude Scores

H6352-Ch08.qxd 7/18/05 3:40 PM Page 155

2. The expressions used in the formula can be defined as follows:

N

1

the total number of participants in group 1

N

2

the total number of participants in group 2

N

x

total number of participants in the group with the largest rank total

T

x

the largest rank total score

3. In order to get data to put in these expressions, you will have to carry

out some calculations on your original data table. When it comes to

ranking the scores, you should rank the scores for both groups as a

single series of ranks. Once you have done this, your data should look

like Table 8.10.

4. Your formula should now look something like this:

5. According to my calculations U 512.

6. The next step is to look up Table A3 in Appendix A to find out if the

result obtained from this data is statistically significant. As we did not

make a prediction about which version of the PDA application partici-

pants would prefer, we are only interested in significance levels for

two-tailed tests.

U

10 10

10 11

2

143

156 Understanding Mobile Human–Computer Interaction

Stylus Input Rank (1) Keypad Entry Rank (2)

Attitude Score Attitude score

(group 1) (group 2)

24 2.5 38 17.5

24 2.5 39 19

32 10.5 37 16

29 5 36 15

38 17.5 34 13

31 8 30 6

27 4 32 10.5

31 8 40 20

21 1 35 14

31 8 33 12

Total 288 68 353 143

Table 8.10 Data Table for Carrying out Mann-Whitney U Calculation

H6352-Ch08.qxd 7/18/05 3:40 PM Page 156

Get *Understanding Mobile Human-Computer Interaction* now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.