Analysis of Multimodality in Interaction

This entry examines key approaches to the analysis of multimodality in interaction, foregrounding the view that human communication is inherently multimodal, embodied, and situated. It begins with Multimodal Conversation Analysis, an extension of the conversation analytic tradition, which integrates gesture, gaze, posture, and artefacts into sequential analysis of talk‐in‐interaction. This approach, developed through the work of scholars such as Goodwin, Kendon, and Mondada, provides fine‐grained accounts of how actions are coordinated through multimodal resources while maintaining emic sensitivity to participants' orientations. The second approach, Multimodal (Inter)action Analysis, originates from mediated discourse theory and advances action as the primary unit of analysis. Building on Scollon and Norris, this framework examines how actions are mediated by semiotic and material means, and introduces concepts such as modal intensity and configuration to account for the dynamic layering of multimodal resources. Applications of this approach in education and workplace communication illustrate its capacity to link micro‐interactional detail to broader cycles of social practice. The third approach, the social semiotic tradition, extends Halliday's systemic functional linguistics and emphasizes meaning‐making as design. Associated with the work of Kress, van Leeuwen, and colleagues, this perspective theorizes modes as socially shaped resources with distinct affordances and ideological dimensions. It highlights multimodality as both analytical and critical, connecting sign‐making practices to wider cultural and institutional contexts. Taken together, these approaches provide complementary insights into the study of multimodal communication and offer robust methodological and theoretical resources for advancing research, pedagogy, and practice in an increasingly complex communicative landscape.

Affordance · Conversation · Conversation analysis · Discourse analysis · Emic and etic · Foregrounding · Multimodal interaction · Multimodality · Semiotics

• A Simplest Systematics for the Organization of Turn-Taking for Conversation

  Open Access•Harvey Sacks, Emanuel A Schegloff et al.•Language•2025

  Add to list