The affective nature of AI-generated news images: Impact on visual journalism

Sejin Paik; Sarah Bonna; Ekaterina Novozhilova; Ge Gao; Jongin Kim; Derry Wijaya; Margrit Betke

doi:10.1109/ACII59096.2023.10388166

The affective nature of AI-generated news images: Impact on visual journalism

Sejin Paik, Sarah Bonna, Ekaterina Novozhilova, Ge Gao, Jongin Kim, Derry Wijaya, Margrit Betke

Data Science Indonesia

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

1 Citation (Scopus)

Abstract

This study explores the affective responses and newsworthiness perceptions of generative AI for visual journalism. While generative AI offers advantages for newsrooms in terms of producing unique images and cutting costs, the potential misuse of AI-generated news images is a cause for concern. For our study, we designed a 3-part news image codebook for affect-labeling news images based on journalism ethics and photography guidelines. We collected 200 news headlines and images retrieved from a variety of U.S. news sources on the topics of gun violence and climate change, generated corresponding news images from DALL-E 2 and asked annotators their emotional responses to the human-selected and AI-generated news images following the codebook. We also examined the impact of modality on emotions by measuring the effects of visual and textual modalities on emotional responses. The findings of this study provide insights into the quality and emotional impact of generative news images produced by humans and AI. Further, results of this work can be useful in developing technical guidelines as well as policy measures for the ethical use of generative AI systems in journalistic production. The codebook, images and annotations are made publicly available to facilitate future research in affective computing, specifically tailored to civic and public-interest journalism.

Original language	English
Title of host publication	2023 11th International Conference on Affective Computing and Intelligent Interaction, ACII 2023
Editors	Shaun Canavan, Dongmei Jiang
Place of Publication	Piscataway NJ USA
Publisher	IEEE, Institute of Electrical and Electronics Engineers
Number of pages	9
ISBN (Electronic)	9798350327434
ISBN (Print)	9798350327441
DOIs	https://doi.org/10.1109/ACII59096.2023.10388166
Publication status	Published - 2023
Event	International Conference on Affective Computing and Intelligent Interaction 2023 - Cambridge, United States of America Duration: 10 Sept 2023 → 13 Sept 2023 Conference number: 11th https://ieeexplore.ieee.org/xpl/conhome/10388070/proceeding (Proceedings) https://acii-conf.net/2023/ (Website)

Conference

Conference	International Conference on Affective Computing and Intelligent Interaction 2023
Abbreviated title	ACII 2023
Country/Territory	United States of America
City	Cambridge
Period	10/09/23 → 13/09/23
Internet address	https://ieeexplore.ieee.org/xpl/conhome/10388070/proceeding (Proceedings) https://acii-conf.net/2023/ (Website)

Keywords

Affective image generation
AI safety
Emotion analysis
Ethics of affective computing
Generative AI
Journalism ethics

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1109/ACII59096.2023.10388166

Cite this

Paik, S., Bonna, S., Novozhilova, E., Gao, G., Kim, J., Wijaya, D., & Betke, M. (2023). The affective nature of AI-generated news images: Impact on visual journalism. In S. Canavan, & D. Jiang (Eds.), 2023 11th International Conference on Affective Computing and Intelligent Interaction, ACII 2023 IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ACII59096.2023.10388166

@inproceedings{ddffab493f5146f39334e0c4e0bd81b1,

title = "The affective nature of AI-generated news images: Impact on visual journalism",

abstract = "This study explores the affective responses and newsworthiness perceptions of generative AI for visual journalism. While generative AI offers advantages for newsrooms in terms of producing unique images and cutting costs, the potential misuse of AI-generated news images is a cause for concern. For our study, we designed a 3-part news image codebook for affect-labeling news images based on journalism ethics and photography guidelines. We collected 200 news headlines and images retrieved from a variety of U.S. news sources on the topics of gun violence and climate change, generated corresponding news images from DALL-E 2 and asked annotators their emotional responses to the human-selected and AI-generated news images following the codebook. We also examined the impact of modality on emotions by measuring the effects of visual and textual modalities on emotional responses. The findings of this study provide insights into the quality and emotional impact of generative news images produced by humans and AI. Further, results of this work can be useful in developing technical guidelines as well as policy measures for the ethical use of generative AI systems in journalistic production. The codebook, images and annotations are made publicly available to facilitate future research in affective computing, specifically tailored to civic and public-interest journalism.",

keywords = "Affective image generation, AI safety, Emotion analysis, Ethics of affective computing, Generative AI, Journalism ethics",

author = "Sejin Paik and Sarah Bonna and Ekaterina Novozhilova and Ge Gao and Jongin Kim and Derry Wijaya and Margrit Betke",

note = "Funding Information: This work is supported in part by the U.S. NSF grant 1838193 and DARPA HR001118S0044 (the LwLL program). The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes. The views and conclusions contained in this publication are those of the authors and should not be interpreted as representing official policies or endorsements of NSF, DARPA, or the U.S. Government. Publisher Copyright: {\textcopyright} 2023 IEEE.; International Conference on Affective Computing and Intelligent Interaction 2023, ACII 2023 ; Conference date: 10-09-2023 Through 13-09-2023",

year = "2023",

doi = "10.1109/ACII59096.2023.10388166",

language = "English",

isbn = "9798350327441",

editor = "Shaun Canavan and Dongmei Jiang",

booktitle = "2023 11th International Conference on Affective Computing and Intelligent Interaction, ACII 2023",

publisher = "IEEE, Institute of Electrical and Electronics Engineers",

address = "United States of America",

url = "https://ieeexplore.ieee.org/xpl/conhome/10388070/proceeding, https://acii-conf.net/2023/",

}

Paik, S, Bonna, S, Novozhilova, E, Gao, G, Kim, J, Wijaya, D & Betke, M 2023, The affective nature of AI-generated news images: Impact on visual journalism. in S Canavan & D Jiang (eds), 2023 11th International Conference on Affective Computing and Intelligent Interaction, ACII 2023. IEEE, Institute of Electrical and Electronics Engineers, Piscataway NJ USA, International Conference on Affective Computing and Intelligent Interaction 2023, Cambridge, Massachusetts, United States of America, 10/09/23. https://doi.org/10.1109/ACII59096.2023.10388166

The affective nature of AI-generated news images: Impact on visual journalism. / Paik, Sejin; Bonna, Sarah; Novozhilova, Ekaterina et al.
2023 11th International Conference on Affective Computing and Intelligent Interaction, ACII 2023. ed. / Shaun Canavan; Dongmei Jiang. Piscataway NJ USA: IEEE, Institute of Electrical and Electronics Engineers, 2023.

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

TY - GEN

T1 - The affective nature of AI-generated news images

T2 - International Conference on Affective Computing and Intelligent Interaction 2023

AU - Paik, Sejin

AU - Bonna, Sarah

AU - Novozhilova, Ekaterina

AU - Gao, Ge

AU - Kim, Jongin

AU - Wijaya, Derry

AU - Betke, Margrit

N1 - Conference code: 11th

PY - 2023

Y1 - 2023

N2 - This study explores the affective responses and newsworthiness perceptions of generative AI for visual journalism. While generative AI offers advantages for newsrooms in terms of producing unique images and cutting costs, the potential misuse of AI-generated news images is a cause for concern. For our study, we designed a 3-part news image codebook for affect-labeling news images based on journalism ethics and photography guidelines. We collected 200 news headlines and images retrieved from a variety of U.S. news sources on the topics of gun violence and climate change, generated corresponding news images from DALL-E 2 and asked annotators their emotional responses to the human-selected and AI-generated news images following the codebook. We also examined the impact of modality on emotions by measuring the effects of visual and textual modalities on emotional responses. The findings of this study provide insights into the quality and emotional impact of generative news images produced by humans and AI. Further, results of this work can be useful in developing technical guidelines as well as policy measures for the ethical use of generative AI systems in journalistic production. The codebook, images and annotations are made publicly available to facilitate future research in affective computing, specifically tailored to civic and public-interest journalism.

AB - This study explores the affective responses and newsworthiness perceptions of generative AI for visual journalism. While generative AI offers advantages for newsrooms in terms of producing unique images and cutting costs, the potential misuse of AI-generated news images is a cause for concern. For our study, we designed a 3-part news image codebook for affect-labeling news images based on journalism ethics and photography guidelines. We collected 200 news headlines and images retrieved from a variety of U.S. news sources on the topics of gun violence and climate change, generated corresponding news images from DALL-E 2 and asked annotators their emotional responses to the human-selected and AI-generated news images following the codebook. We also examined the impact of modality on emotions by measuring the effects of visual and textual modalities on emotional responses. The findings of this study provide insights into the quality and emotional impact of generative news images produced by humans and AI. Further, results of this work can be useful in developing technical guidelines as well as policy measures for the ethical use of generative AI systems in journalistic production. The codebook, images and annotations are made publicly available to facilitate future research in affective computing, specifically tailored to civic and public-interest journalism.

KW - Affective image generation

KW - AI safety

KW - Emotion analysis

KW - Ethics of affective computing

KW - Generative AI

KW - Journalism ethics

UR - http://www.scopus.com/inward/record.url?scp=85184665586&partnerID=8YFLogxK

U2 - 10.1109/ACII59096.2023.10388166

DO - 10.1109/ACII59096.2023.10388166

M3 - Conference Paper

AN - SCOPUS:85184665586

SN - 9798350327441

BT - 2023 11th International Conference on Affective Computing and Intelligent Interaction, ACII 2023

A2 - Canavan, Shaun

A2 - Jiang, Dongmei

PB - IEEE, Institute of Electrical and Electronics Engineers

CY - Piscataway NJ USA

Y2 - 10 September 2023 through 13 September 2023

ER -

Paik S, Bonna S, Novozhilova E, Gao G, Kim J, Wijaya D et al. The affective nature of AI-generated news images: Impact on visual journalism. In Canavan S, Jiang D, editors, 2023 11th International Conference on Affective Computing and Intelligent Interaction, ACII 2023. Piscataway NJ USA: IEEE, Institute of Electrical and Electronics Engineers. 2023 doi: 10.1109/ACII59096.2023.10388166

The affective nature of AI-generated news images: Impact on visual journalism

Abstract

Conference

Keywords

UN SDGs

Access to Document

Other files and links

Cite this