What’s in a click? A social semiotic framework for the multimodal analysis of website interactivity

Abstract

This article presents a social semiotic framework for the multimodal analysis of website interactivity. Distinguishing it from interaction, it defines interactivity as the affordance of a text of being acted (up)on, thus including hypertextuality. The author introduces the notion of ‘interactive sites/signs’ as the loci of interactivity in digital texts; these have a two-fold nature and a two-dimensional functioning. In their two-fold nature, they are both places enabling actions producing effects and forms endowed with meanings. Notwithstanding the non-direct correspondence between forms, actions and effects (which makes any specific association between the three significant within a webpage design), and in spite of their many possible forms (encompassing still and dynamic images, shapes and writing), a small range of actions can activate them (click/click+type/hover), producing a restricted set of textual effects (access/provide/transfer text). In their two-dimensional functioning, interactive sites/signs function both syntagmatically, on the page where they are displayed, in their relation with other co-occurring elements, and paradigmatically, opening to optional text realizations, hence in their relation with these. The framework adapts Halliday’s three metafunctions to the analysis of the two-fold nature and two-dimensional functioning of interactive sites/signs. It provides a fine-grained account of the interactive meaning potentials of digital texts, distinguishing between a text’s aesthetics of interactivity – as visually communicated before it is activated, performed and experienced – and its functionality, in the configuration of interactive possibilities offered by a page. Designed to complement the extant practices of text analysis of webpages, the framework can be used comparatively, as exemplified in its application to the analysis of two blog pages, and can provide a more refined assessment of the interactive meaning potential of a webpage than traditional methodologies such as content analysis.