META: Text Detoxification by leveraging METAmorphic Relations and Deep Learning Methods

Alika Choo, Arghya Pal, Sailaja Rajanala, Arkendu Sen

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

Abstract

In the world of online interactions, social communities face a significant challenge: the spread of offensive content and hate speech through toxic languages. Such issues led to growing research on text detoxification systems that can automatically rewrite toxic content. A systematic evaluation is required to ensure these systems produce high-quality detoxified text that modifies the original text to be non-toxic while preserving its content. However, this often relies on large amounts of labelled data and human judgement, which may not always be feasible. This limitation is typically known as the oracle problem. Metamorphic testing (MT) has conventionally been used to solve the oracle problem by deriving metamorphic relations (MRs) to test a program's functionality. A new MT approach focused on data validation showed that MRs incorporated with tools can be used to identify defects in machine translation services. This paper draws inspiration from this new MT perspective by presenting four metamorphic relations incorporated with tools to evaluate style transfer accuracy, content preservation, fluency, and a joint of these three. Our proposed approach effectively identifies defective behaviour in state-of-the-art text detoxification systems.

Original languageEnglish
Title of host publication2024 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC 2024)
EditorsYi Wang, Liming Zhang, Chi Man Pun, Jinyu Tian
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages1763-1768
Number of pages6
ISBN (Electronic)9798350367331
ISBN (Print)9798350367348
DOIs
Publication statusPublished - 2024
EventAnnual Summit and Conference of the Asia-Pacific-Signal-and-Information-Processing-Association (APSIPA) 2024 - Macau, China
Duration: 3 Dec 20246 Dec 2024
https://ieeexplore.ieee.org/xpl/conhome/10848542/proceeding (Proceedings)
https://www.apsipa2024.org/ (Website)
http://www.apsipa2024.org/ (Website)

Publication series

NameAsia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC
PublisherIEEE, Institute of Electrical and Electronics Engineers
ISSN (Print)2640-009X
ISSN (Electronic)2640-0103

Conference

ConferenceAnnual Summit and Conference of the Asia-Pacific-Signal-and-Information-Processing-Association (APSIPA) 2024
Abbreviated titleAPSIPA ASC 2024
Country/TerritoryChina
CityMacau
Period3/12/246/12/24
Internet address

Cite this