P1: RAZI
December 18, 2014 12:47 K24618 K24618˙Book
2
Semiotic Foundations of
HCI and UX Design
As we have described in our thinking and acting, natural language plays a central
part. This language defines a structure even before we form something and can be
regarded as the architecture of design. Our consciousness is the result of language
informing design. Grammar allows for many combinations of objects and actions,
but ideology establishes the privileged connection of the two of what is correct and
possible. In order to set forth the privileged connection, ideology employs different
forms of persuasion. To grasp the expression of these structures in HCI, we chose the
perspective of linguistics and semiotics.
By semiotics, we mean a theory of signs. We combine the Anglo-American semi-
otics (semeiotics) perspective with the French semiology (“s´emiologie”) approach
(Barthes, 1977). According to Peirce, a sign is “something that stands for someone
or something in some respect or capacity” (Peirce in Buchler, 1955, p. 99). Four
dimensions form the sign: lexical (Eco, 1979), syntactics, semantics, and pragmatics
(Morris, 1970).
The lexical dimension (Eco, 1979) focuses on the generation of elements and their
syntax. The lexical dimension is bound to the physical constraints (see Norman, 1999)
of the UI, such as display size, resolution, and material used. The lexical dimension
is related to consumption (Marcus, 2009).
Syntactics is “the study of the syntactical relations of signs to one another in
abstraction from the relations of signs to objects or to interepreters... (Morris, 1970,
p. 13). In this dimension we deal with the grammar constituting relations between the
perceivable elements or sign vehicles.
Semantics, on the other hand, “deals with the relation of signs to their designata and
so to the objects which they may or do denote” (Morris, 1970, p. 21). This dimension is
devoted to the relation between vehiculae and the object, content, action, or “meaning”
the UI represents and enables. This dimension is connected to mental models (in the
sense of which functions the system allows) and affordances (Norman, 2002; Gibson,
1977).
Pragmatics “deals with the biotic aspects of semiosis, that is, with all the psy-
chological, biological, and sociological phenomena which occur in the functioning
of signs” (Morris, 1970, p. 30). This most complex dimension focuses on how we
use or interpret the vehicula-object relation, that is, what is the sign’s purpose? The
pragmatic dimension governs how signs are used, or understood in their conventional
and symbolic form.
We consider semiotics as a foundation of interaction and communication design
in HCI, because it is concerned with meaningful arrangement of UI elements across
space and time. The semiotics perspective in the contextof HCI is increasingly popular
in presenting a different approach. The classical linguistic and semiotic foundations of
HCI were previously set down by, e.g., Andersen, 1997, 2001; Brandt, 1993; Nadin,
9

Get Cross-Cultural Human-Computer Interaction and User Experience Design now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.