Exploring textual and speech information in dialogue act classification with speaker domain adaptation

Xuanli He, Quan Hung Tran, William Havard, Laurent Besacier, Ingrid Zukerman, Reza Haffari

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

Abstract

In spite of the recent success of Dialogue Act (DA) classification, the majority of prior works focus on text-based classification with oracle transcriptions, i.e. human transcriptions, instead of Automatic Speech Recognition (ASR)’s transcriptions. Moreover, the performance of this classification task, because of speaker domain shift, may deteriorate. In this paper, we explore the effectiveness of using both acoustic and textual signals, either oracle or ASR transcriptions, and investigate speaker domain adaptation for DA classification. Our multimodal model proves to be superior to the unimodal models, particularly when the oracle transcriptions are not available. We also propose an effective method for speaker domain adaptation, which achieves competitive results.
Original languageEnglish
Title of host publicationAustralasian Language Technology Association Workshop 2018 - Proceedings of the Workshop
Subtitle of host publication10–12 December 2018 The University of Otago Dunedin, New Zealand
EditorsSunghwan Mac Kim, Xiuzhen (Jenny) Zhang
Place of PublicationAustralia
PublisherAustralian Language Technology Association (ALTA)
Number of pages5
Publication statusPublished - 2018
EventAnnual Workshop of The Australasian Language Technology Association 2018 - Dunedin, New Zealand
Duration: 10 Dec 201812 Dec 2018
Conference number: 16th
http://alta2018.alta.asn.au/submissions.html

Conference

ConferenceAnnual Workshop of The Australasian Language Technology Association 2018
Abbreviated titleALTA 2018
CountryNew Zealand
CityDunedin
Period10/12/1812/12/18
Internet address

Keywords

  • Dialogue acts
  • Domain adaptation
  • Multimodal model

Cite this

He, X., Tran, Q. H., Havard, W., Besacier, L., Zukerman, I., & Haffari, R. (2018). Exploring textual and speech information in dialogue act classification with speaker domain adaptation. In S. Mac Kim, & X. J. Zhang (Eds.), Australasian Language Technology Association Workshop 2018 - Proceedings of the Workshop: 10–12 December 2018 The University of Otago Dunedin, New Zealand Australia: Australian Language Technology Association (ALTA).
He, Xuanli ; Tran, Quan Hung ; Havard, William ; Besacier, Laurent ; Zukerman, Ingrid ; Haffari, Reza. / Exploring textual and speech information in dialogue act classification with speaker domain adaptation. Australasian Language Technology Association Workshop 2018 - Proceedings of the Workshop: 10–12 December 2018 The University of Otago Dunedin, New Zealand. editor / Sunghwan Mac Kim ; Xiuzhen (Jenny) Zhang. Australia : Australian Language Technology Association (ALTA), 2018.
@inproceedings{52d0ef0f5d23419cb79bc5a315ac6834,
title = "Exploring textual and speech information in dialogue act classification with speaker domain adaptation",
abstract = "In spite of the recent success of Dialogue Act (DA) classification, the majority of prior works focus on text-based classification with oracle transcriptions, i.e. human transcriptions, instead of Automatic Speech Recognition (ASR)’s transcriptions. Moreover, the performance of this classification task, because of speaker domain shift, may deteriorate. In this paper, we explore the effectiveness of using both acoustic and textual signals, either oracle or ASR transcriptions, and investigate speaker domain adaptation for DA classification. Our multimodal model proves to be superior to the unimodal models, particularly when the oracle transcriptions are not available. We also propose an effective method for speaker domain adaptation, which achieves competitive results.",
keywords = "Dialogue acts, Domain adaptation, Multimodal model",
author = "Xuanli He and Tran, {Quan Hung} and William Havard and Laurent Besacier and Ingrid Zukerman and Reza Haffari",
year = "2018",
language = "English",
editor = "{Mac Kim}, {Sunghwan } and Zhang, {Xiuzhen (Jenny) }",
booktitle = "Australasian Language Technology Association Workshop 2018 - Proceedings of the Workshop",
publisher = "Australian Language Technology Association (ALTA)",

}

He, X, Tran, QH, Havard, W, Besacier, L, Zukerman, I & Haffari, R 2018, Exploring textual and speech information in dialogue act classification with speaker domain adaptation. in S Mac Kim & XJ Zhang (eds), Australasian Language Technology Association Workshop 2018 - Proceedings of the Workshop: 10–12 December 2018 The University of Otago Dunedin, New Zealand. Australian Language Technology Association (ALTA), Australia, Annual Workshop of The Australasian Language Technology Association 2018, Dunedin, New Zealand, 10/12/18.

Exploring textual and speech information in dialogue act classification with speaker domain adaptation. / He, Xuanli; Tran, Quan Hung; Havard, William; Besacier, Laurent; Zukerman, Ingrid; Haffari, Reza.

Australasian Language Technology Association Workshop 2018 - Proceedings of the Workshop: 10–12 December 2018 The University of Otago Dunedin, New Zealand. ed. / Sunghwan Mac Kim; Xiuzhen (Jenny) Zhang. Australia : Australian Language Technology Association (ALTA), 2018.

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

TY - GEN

T1 - Exploring textual and speech information in dialogue act classification with speaker domain adaptation

AU - He, Xuanli

AU - Tran, Quan Hung

AU - Havard, William

AU - Besacier, Laurent

AU - Zukerman, Ingrid

AU - Haffari, Reza

PY - 2018

Y1 - 2018

N2 - In spite of the recent success of Dialogue Act (DA) classification, the majority of prior works focus on text-based classification with oracle transcriptions, i.e. human transcriptions, instead of Automatic Speech Recognition (ASR)’s transcriptions. Moreover, the performance of this classification task, because of speaker domain shift, may deteriorate. In this paper, we explore the effectiveness of using both acoustic and textual signals, either oracle or ASR transcriptions, and investigate speaker domain adaptation for DA classification. Our multimodal model proves to be superior to the unimodal models, particularly when the oracle transcriptions are not available. We also propose an effective method for speaker domain adaptation, which achieves competitive results.

AB - In spite of the recent success of Dialogue Act (DA) classification, the majority of prior works focus on text-based classification with oracle transcriptions, i.e. human transcriptions, instead of Automatic Speech Recognition (ASR)’s transcriptions. Moreover, the performance of this classification task, because of speaker domain shift, may deteriorate. In this paper, we explore the effectiveness of using both acoustic and textual signals, either oracle or ASR transcriptions, and investigate speaker domain adaptation for DA classification. Our multimodal model proves to be superior to the unimodal models, particularly when the oracle transcriptions are not available. We also propose an effective method for speaker domain adaptation, which achieves competitive results.

KW - Dialogue acts

KW - Domain adaptation

KW - Multimodal model

M3 - Conference Paper

BT - Australasian Language Technology Association Workshop 2018 - Proceedings of the Workshop

A2 - Mac Kim, Sunghwan

A2 - Zhang, Xiuzhen (Jenny)

PB - Australian Language Technology Association (ALTA)

CY - Australia

ER -

He X, Tran QH, Havard W, Besacier L, Zukerman I, Haffari R. Exploring textual and speech information in dialogue act classification with speaker domain adaptation. In Mac Kim S, Zhang XJ, editors, Australasian Language Technology Association Workshop 2018 - Proceedings of the Workshop: 10–12 December 2018 The University of Otago Dunedin, New Zealand. Australia: Australian Language Technology Association (ALTA). 2018