Can Prosody Aid the Automatic Classification of Dialog Acts in Conversational Speech

Bibliographic Data

ID10255243
AuthorsElizabeth Shriberg (0009-0004-3779-4956, SRI International, Menlo Park, CA, corresponding author), Andreas Stolcke (0000-0002-9925-905X, SRI International, Menlo Park, CA), Daniel Jurafsky (University of Colorado Boulder), Noah Coccaro (University of Colorado Boulder), Marie Meteer (BBN Systems and Technologies, Cambridge, MA), Rebecca Bates (0000-0003-2553-6066, Boston University), Paul Taylor (0000-0002-8900-2912, University of Edinburgh), Klaus Ries (Carnegie Mellon University, Pittsburgh), Rachel Martin (0000-0002-2865-9881, Johns Hopkins University, Baltimore), Rachel W Martin (0000-0001-9996-7411, Johns Hopkins University), Carol van Ess-Dykema (U.S. Department of Defense, Ft. Meade, MD)
Year1998
Volume41
Issue3-4
Pages443-492
Publication date1998-07-01
Peer ReviewedYes
Open AccessYes
TypeARTICLE
VenueLanguage and Speech (JOURNAL)
Journal identifiersISSN: 0023-8309 • E-ISSN: 1756-6053
PublisherSAGE Publications (PUBLISHER • US)
DOI10.1177/002383099804100410
PMID10746366
OpenAlexW2112078429
LanguageEN
Citations received14
References cited50

Identifying whether an utterance is a statement, question, greeting, and so forth is integral to effective automatic understanding of natural dialog. Little is known, however, about how such dialog acts (DAs) can be automatically classified in truly natural conversation. This study asks whether current approaches, which use mainly word information, could be improved by adding prosodic information. The study is based on more than 1000 conversations from the Switchboard corpus. DAs were hand-annotated, and prosodic features (duration, pause, F0, energy, and speaking rate) were automatically extracted for each DA. In training, decision trees based on these features were inferred; trees were then applied to unseen test data to evaluate performance. Performance was evaluated for prosody models alone, and after combining the prosody models with word information—either from true words or from the output of an automatic speech recognizer. For an overall classification task, as well as three subtasks, prosody made significant contributions to classification. Feature-specific analyses further revealed that although canonical features (such as F0 for questions) were important, less obvious features could compensate if canonical features were removed. Finally, in each task, integrating the prosodic model with a DA-specific statistical language model improved performance over that of the language model alone, especially for the case of recognized words. Results suggest that DAs are redundantly marked in natural conversation, and that a variety of automatically extractable prosodic features could aid dialog processing in speech applications

Conversation · Dialog box · Feature (linguistics · Linguistics · Natural language · Natural language processing · Prosody · Speech recognition · Task (project management · Utterance · Variety (cybernetics · Artificial Intelligence · Communication · Computer Science · Natural Language Processing Techniques · Psychology · Speech and dialogue systems · Topic Modeling

Tools

Open DOISci-HubOpen Access