Chiao-I Tseng
Biographic Data
| ID | 166932 |
|---|---|
| NAME | Chiao-I Tseng |
| GIVEN NAMES | Chiao-I |
| FAMILY NAME | Tseng |
| SIGNATURE | TSENG C |
| AFFILIATIONS | University of Bremen |
| ORCID | 0000-0001-6902-7322 |
| VERIFIED | Yes |
| TOTAL WORKS | 19 |
| TOTAL CITATIONS | 15 |
| WORKS AS AUTHOR | 19 |
| WORKS AS EDITOR | 0 |
| FIRST PUBLICATION YEAR | 2012 |
| LATEST PUBLICATION YEAR | 2026 |
| H-INDEX | 2 |
Bateman, John
John A. Bateman (b. 1957) has a distinguished research background in applied linguistics and has contributed to a wide range of research areas since the 1980s. He started working on natural language generation in 1982 and was awarded the degree of PhD in artificial intelligence in 1986 at Edinburgh University. Since then, he has engaged in research projects on language generation, translation science, discourse structures, and descriptive linguis…
Contrasting public, market-driven and populist news videos on Covid-19
This paper introduces a novel method for investigating how public service, market-driven, and populist news outlets differ in their construction of ideological positions in audiovisual reporting. A semi-automatic framework of multimodal semiotic content analysis is presented that combines computational audiovisual feature detection with theory-driven manual annotation of discourse and narrative elements and shows early results concerning the cons…
Multimodal semiotic content analysis
This article proposes a mixed-method framework for multimodal semiotic content analysis that systematically integrates qualitative semiotic coding with quantitative large-scale data analysis. Building on earlier frameworks by Bell and Milic (‘Goffman’s gender advertisements revisited: combining content analysis with semiotic analysis’, in Visual Communication , 2002) and Serafini and Reid (‘Multimodal content analysis: expanding analytical approa…
Multimodal Cohesion
Multimodal cohesion extends the notion of cohesion originally developed for verbal language so as to apply to communicative artefacts or performances constituted by any combination of semiotic systems, such as static and dynamic images, diagrams, gestures, and sounds. The basic tenet of multimodal cohesion is that recurrences of images, sounds, verbal language, written language, camera movements, framings, musical motifs, sounds, shapes, colors, …
The fragmentation and plasticity of space in film
This paper explores the plasticity of represented space in film by focusing on how space in film can be constructed out of fragments. Motivated by previous research suggesting that the fragmentation of space in film can be hidden from viewers, we propose a framework for analysing core features of space construction in terms of three main dimensions: 1. Explicitness, or whether a space is shown visually and explicit, or only implied through sound;…
Children’s comprehension of time and their interpretation of event relations in narrative film
Audiovisual narratives are widely adopted as a resource for promoting children’s learning and critical multimodal literacy, which involves considering the social themes presented in multimodal texts. Previous research reveals that storytelling that employs achronological sequencing challenges children’s temporal cognition and narrative comprehension. Little is known, however, about how children’s comprehension of events relates to their interpret…
Multimodal cohesion and viewers' comprehension of scene transitions in film
This paper presents three empirical studies that unravel how the devices of multimodal cohesion support viewers' narrative interpretation of scene transitions in film. The linguistics-informed method of cohesion analysis in film uncovers the establishment of cohesive ties between characters, objects, settings and characters' actions. Previous studies using eye-tracking and comprehension tests already indicate the significance of multimodal cohesi…
Children’s comprehension of time in audiovisual narratives
Actions and digital empathy in the interactive storytelling of serious games
This paper proposes a multimodal discourse method for analysing empathy-related actions in the interactive narratives of serious games. It first reviews recent theories of socially impactful designs in serious games, particularly the debates about whether serious games should focus on actions of tackling social issues or on the player’s emotional engagement. This paper then sets out a multimodal discourse method of event action patterns, which br…
Children’s experiences with a transmedia narrative
The impact of multimodal cohesion on attention and interpretation in film
Harnessing the potential of transmedia narratives for critical multimodal literacy
Literary narratives are well recognised for their power to foster engagement with complex social themes. Transmedia narratives, which present the same story in different media, can help advance both critical multimodal discourse studies and multiliteracies pedagogies. To harness this potential, we need to develop methods for systematically relating media affordances to discourse-semantic patterns and the broad social themes these patterns constru…
Multimodality
This textbook provides the first foundational introduction to the practice of analysing multimodality, covering the full breadth of media and situations in which multimodality needs to be a concern. Readers learn via use cases how to approach any multimodal situation and to derive their own specifically tailored sets of methods for conducting and evaluating analyses. Extensive references and critical discussion of existing approaches from many di…
Beyond the media boundaries
Towards next-generation visual archives
The digital turn in visual studies has played a major role in the terminological overlap between ‘archive’, ‘database’ and ‘corpus’, and it has brought about a number of positive developments such as improved accessibility and availability. At the same time, it has also raised important questions pertaining to the materiality, searchability, annotation and analysis of the data at hand. Through a series of theoretical constructs and empirical exam…
Revisiting dynamic space in film from a semiotic perspective
This paper revisits a particular area of concern in computer generated (CG) visual effects, namely, the problem of the space in dynamic, multiple panels created by digital processes. The paper critiques recent statements made on narrative understanding, positing equivalences between a viewer’s navigation across dynamic frames in CG images and human-computer interactions, as well as claims of narrative complexity in films using dynamic frames. Thi…
Analysing characters' interactions in filmic text
This paper presents an analytical tool for constructing characters' action patterns in film. Understanding characters in moving images has been regarded as one of the most significant elements of narrative comprehension and interpretation because spectators' inferences of narrative structures are substantially mediated by characters. The social semiotic method presented in this paper shows how characters' actions and interactions can be systemati…
Multimodal narrative construction in Christopher Nolan’s Memento a description of analytic method
In this article, the authors propose a systematic method for analysing film that bridges the gap between analyses of narrative concerns, on the one hand, and the analysis of fine-grained technical details, on the other. They illustrate the approach on the basis of an analysis of Memento, a film particularly noted for its unusual narrative structure and show how their method maintains a tighter linkage with the filmic material presented while at t…
Audiovisual texture in scene transition
This paper shows how the method of cohesion analysis in film provides a strong foundation for constructing coherent filmic narrative across different spatial and temporal settings. Until now the unit of a fine-grained systematic analysis of film viewers' narrative comprehension has been anchored in the shot. This paper provides a semiotic descriptive tool of filmic cohesion for analyzing elements within and across shots, i.e., how viewers compreh…
Analysing characters' interactions in filmic text
This paper presents an analytical tool for constructing characters' action patterns in film. Understanding characters in moving images has been regarded as one of the most significant elements of narrative comprehension and interpretation because spectators' inferences of narrative structures are substantially mediated by characters. The social semiotic method presented in this paper shows how characters' actions and interactions can be systemati…
Harnessing the potential of transmedia narratives for critical multimodal literacy
Literary narratives are well recognised for their power to foster engagement with complex social themes. Transmedia narratives, which present the same story in different media, can help advance both critical multimodal discourse studies and multiliteracies pedagogies. To harness this potential, we need to develop methods for systematically relating media affordances to discourse-semantic patterns and the broad social themes these patterns constru…
The impact of multimodal cohesion on attention and interpretation in film
Multimodal narrative construction in Christopher Nolan’s Memento a description of analytic method
In this article, the authors propose a systematic method for analysing film that bridges the gap between analyses of narrative concerns, on the one hand, and the analysis of fine-grained technical details, on the other. They illustrate the approach on the basis of an analysis of Memento, a film particularly noted for its unusual narrative structure and show how their method maintains a tighter linkage with the filmic material presented while at t…
Audiovisual texture in scene transition
This paper shows how the method of cohesion analysis in film provides a strong foundation for constructing coherent filmic narrative across different spatial and temporal settings. Until now the unit of a fine-grained systematic analysis of film viewers' narrative comprehension has been anchored in the shot. This paper provides a semiotic descriptive tool of filmic cohesion for analyzing elements within and across shots, i.e., how viewers compreh…
Multimodal narrative construction in Christopher Nolan’s Memento a description of analytic method
In this article, the authors propose a systematic method for analysing film that bridges the gap between analyses of narrative concerns, on the one hand, and the analysis of fine-grained technical details, on the other. They illustrate the approach on the basis of an analysis of Memento, a film particularly noted for its unusual narrative structure and show how their method maintains a tighter linkage with the filmic material presented while at t…
Audiovisual texture in scene transition
This paper shows how the method of cohesion analysis in film provides a strong foundation for constructing coherent filmic narrative across different spatial and temporal settings. Until now the unit of a fine-grained systematic analysis of film viewers' narrative comprehension has been anchored in the shot. This paper provides a semiotic descriptive tool of filmic cohesion for analyzing elements within and across shots, i.e., how viewers compreh…
Analysing characters' interactions in filmic text
This paper presents an analytical tool for constructing characters' action patterns in film. Understanding characters in moving images has been regarded as one of the most significant elements of narrative comprehension and interpretation because spectators' inferences of narrative structures are substantially mediated by characters. The social semiotic method presented in this paper shows how characters' actions and interactions can be systemati…
Towards next-generation visual archives
The digital turn in visual studies has played a major role in the terminological overlap between ‘archive’, ‘database’ and ‘corpus’, and it has brought about a number of positive developments such as improved accessibility and availability. At the same time, it has also raised important questions pertaining to the materiality, searchability, annotation and analysis of the data at hand. Through a series of theoretical constructs and empirical exam…
Revisiting dynamic space in film from a semiotic perspective
This paper revisits a particular area of concern in computer generated (CG) visual effects, namely, the problem of the space in dynamic, multiple panels created by digital processes. The paper critiques recent statements made on narrative understanding, positing equivalences between a viewer’s navigation across dynamic frames in CG images and human-computer interactions, as well as claims of narrative complexity in films using dynamic frames. Thi…
Multimodality
This textbook provides the first foundational introduction to the practice of analysing multimodality, covering the full breadth of media and situations in which multimodality needs to be a concern. Readers learn via use cases how to approach any multimodal situation and to derive their own specifically tailored sets of methods for conducting and evaluating analyses. Extensive references and critical discussion of existing approaches from many di…
Beyond the media boundaries
Children’s experiences with a transmedia narrative
The impact of multimodal cohesion on attention and interpretation in film
Harnessing the potential of transmedia narratives for critical multimodal literacy
Literary narratives are well recognised for their power to foster engagement with complex social themes. Transmedia narratives, which present the same story in different media, can help advance both critical multimodal discourse studies and multiliteracies pedagogies. To harness this potential, we need to develop methods for systematically relating media affordances to discourse-semantic patterns and the broad social themes these patterns constru…
Actions and digital empathy in the interactive storytelling of serious games
This paper proposes a multimodal discourse method for analysing empathy-related actions in the interactive narratives of serious games. It first reviews recent theories of socially impactful designs in serious games, particularly the debates about whether serious games should focus on actions of tackling social issues or on the player’s emotional engagement. This paper then sets out a multimodal discourse method of event action patterns, which br…
Children’s comprehension of time in audiovisual narratives
Multimodal cohesion and viewers' comprehension of scene transitions in film
This paper presents three empirical studies that unravel how the devices of multimodal cohesion support viewers' narrative interpretation of scene transitions in film. The linguistics-informed method of cohesion analysis in film uncovers the establishment of cohesive ties between characters, objects, settings and characters' actions. Previous studies using eye-tracking and comprehension tests already indicate the significance of multimodal cohesi…
Multimodal Cohesion
Multimodal cohesion extends the notion of cohesion originally developed for verbal language so as to apply to communicative artefacts or performances constituted by any combination of semiotic systems, such as static and dynamic images, diagrams, gestures, and sounds. The basic tenet of multimodal cohesion is that recurrences of images, sounds, verbal language, written language, camera movements, framings, musical motifs, sounds, shapes, colors, …
The fragmentation and plasticity of space in film
This paper explores the plasticity of represented space in film by focusing on how space in film can be constructed out of fragments. Motivated by previous research suggesting that the fragmentation of space in film can be hidden from viewers, we propose a framework for analysing core features of space construction in terms of three main dimensions: 1. Explicitness, or whether a space is shown visually and explicit, or only implied through sound;…
Children’s comprehension of time and their interpretation of event relations in narrative film
Audiovisual narratives are widely adopted as a resource for promoting children’s learning and critical multimodal literacy, which involves considering the social themes presented in multimodal texts. Previous research reveals that storytelling that employs achronological sequencing challenges children’s temporal cognition and narrative comprehension. Little is known, however, about how children’s comprehension of events relates to their interpret…
Bateman, John
John A. Bateman (b. 1957) has a distinguished research background in applied linguistics and has contributed to a wide range of research areas since the 1980s. He started working on natural language generation in 1982 and was awarded the degree of PhD in artificial intelligence in 1986 at Edinburgh University. Since then, he has engaged in research projects on language generation, translation science, discourse structures, and descriptive linguis…
Contrasting public, market-driven and populist news videos on Covid-19
This paper introduces a novel method for investigating how public service, market-driven, and populist news outlets differ in their construction of ideological positions in audiovisual reporting. A semi-automatic framework of multimodal semiotic content analysis is presented that combines computational audiovisual feature detection with theory-driven manual annotation of discourse and narrative elements and shows early results concerning the cons…
Multimodal semiotic content analysis
This article proposes a mixed-method framework for multimodal semiotic content analysis that systematically integrates qualitative semiotic coding with quantitative large-scale data analysis. Building on earlier frameworks by Bell and Milic (‘Goffman’s gender advertisements revisited: combining content analysis with semiotic analysis’, in Visual Communication , 2002) and Serafini and Reid (‘Multimodal content analysis: expanding analytical approa…
Computer Science (12 works) · Linguistics (12 works) · Narrative (12 works) · Semiotics (9 works) · Art (8 works) · Philosophy (8 works) · Sociology (8 works) · Digital Games and Media (7 works) · Multimodality (7 works) · Psychology (7 works)