Sentiment Analysis using DistilBERT

Song Yi Ng; Kian Ming Lim; Chin Poo Lee; Jit Yan Lim

doi:10.1109/ICSPC59664.2023.10420272

Sentiment Analysis using DistilBERT

Song Yi Ng
, Kian Ming Lim
, Chin Poo Lee
, Jit Yan Lim

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research

7 Citations (Scopus)

Abstract

Transformers is an architecture that performs well in NLP task. To understand and improve its performance on sentiment analysis, DistilBERT is employed as the base model. Sentiment analysis is a process that extracts subjective information from textual data and categorizes them into different classes. The classification classes may include polarity (positive, neutral, negative) or emotions (happy, sad, angry). In addition, multiple techniques such as fine tuning, regularization and hyperparameter tuning are applied to improve the performance of the model. The proposed solution acquired an accuracy score of 85.41% on Internet Movie Database (IMDB) dataset and 86.59% on Customer Reviews (CR) dataset.

Original language	English
Title of host publication	2023 IEEE 11th Conference on Systems, Process & Control (ICSPC) - Conference Proceedings
Editors	Ramli Adnan
Place of Publication	Piscataway NJ USA
Publisher	IEEE, Institute of Electrical and Electronics Engineers
Pages	84-89
Number of pages	6
ISBN (Electronic)	9798350340860, 9798350340853
ISBN (Print)	9798350340877
DOIs	https://doi.org/10.1109/ICSPC59664.2023.10420272
Publication status	Published - 2023
Externally published	Yes
Event	IEEE Conference on Systems, Process and Control (ICSPC) 2023 - Hatten Hotel, Malacca, Malaysia Duration: 16 Dec 2023 → 16 Dec 2023 Conference number: 11th https://ieeexplore.ieee.org/xpl/conhome/10419865/proceeding (Published proceedings) https://sites.google.com/view/icspc/home (Website)

Conference

Conference	IEEE Conference on Systems, Process and Control (ICSPC) 2023
Abbreviated title	ICSPC 2023
Country/Territory	Malaysia
City	Malacca
Period	16/12/23 → 16/12/23
Internet address	https://ieeexplore.ieee.org/xpl/conhome/10419865/proceeding (Published proceedings) https://sites.google.com/view/icspc/home (Website)

Keywords

Deep Learning
DistilBERT
Sentiment Analysis
Transformers

Access to Document

10.1109/ICSPC59664.2023.10420272

Cite this

@inproceedings{d834cf140f5245b4ab4aa07a91cd80f4,

title = "Sentiment Analysis using DistilBERT",

abstract = "Transformers is an architecture that performs well in NLP task. To understand and improve its performance on sentiment analysis, DistilBERT is employed as the base model. Sentiment analysis is a process that extracts subjective information from textual data and categorizes them into different classes. The classification classes may include polarity (positive, neutral, negative) or emotions (happy, sad, angry). In addition, multiple techniques such as fine tuning, regularization and hyperparameter tuning are applied to improve the performance of the model. The proposed solution acquired an accuracy score of 85.41\% on Internet Movie Database (IMDB) dataset and 86.59\% on Customer Reviews (CR) dataset.",

keywords = "Deep Learning, DistilBERT, Sentiment Analysis, Transformers",

author = "Ng, \{Song Yi\} and Lim, \{Kian Ming\} and Lee, \{Chin Poo\} and Lim, \{Jit Yan\}",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; IEEE Conference on Systems, Process and Control (ICSPC) 2023, ICSPC 2023 ; Conference date: 16-12-2023 Through 16-12-2023",

year = "2023",

doi = "10.1109/ICSPC59664.2023.10420272",

language = "English",

isbn = "9798350340877",

pages = "84--89",

editor = "Ramli Adnan",

booktitle = "2023 IEEE 11th Conference on Systems, Process \& Control (ICSPC) - Conference Proceedings",

publisher = "IEEE, Institute of Electrical and Electronics Engineers",

address = "United States of America",

url = "https://ieeexplore.ieee.org/xpl/conhome/10419865/proceeding, https://sites.google.com/view/icspc/home",

}

Ng, SY, Lim, KM, Lee, CP & Lim, JY 2023, Sentiment Analysis using DistilBERT. in R Adnan (ed.), 2023 IEEE 11th Conference on Systems, Process & Control (ICSPC) - Conference Proceedings. IEEE, Institute of Electrical and Electronics Engineers, Piscataway NJ USA, pp. 84-89, IEEE Conference on Systems, Process and Control (ICSPC) 2023, Malacca, Malaysia, 16/12/23. https://doi.org/10.1109/ICSPC59664.2023.10420272

Sentiment Analysis using DistilBERT. / Ng, Song Yi; Lim, Kian Ming; Lee, Chin Poo et al.
2023 IEEE 11th Conference on Systems, Process & Control (ICSPC) - Conference Proceedings. ed. / Ramli Adnan. Piscataway NJ USA: IEEE, Institute of Electrical and Electronics Engineers, 2023. p. 84-89.

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research

TY - GEN

T1 - Sentiment Analysis using DistilBERT

AU - Ng, Song Yi

AU - Lim, Kian Ming

AU - Lee, Chin Poo

AU - Lim, Jit Yan

N1 - Conference code: 11th

PY - 2023

Y1 - 2023

N2 - Transformers is an architecture that performs well in NLP task. To understand and improve its performance on sentiment analysis, DistilBERT is employed as the base model. Sentiment analysis is a process that extracts subjective information from textual data and categorizes them into different classes. The classification classes may include polarity (positive, neutral, negative) or emotions (happy, sad, angry). In addition, multiple techniques such as fine tuning, regularization and hyperparameter tuning are applied to improve the performance of the model. The proposed solution acquired an accuracy score of 85.41% on Internet Movie Database (IMDB) dataset and 86.59% on Customer Reviews (CR) dataset.

AB - Transformers is an architecture that performs well in NLP task. To understand and improve its performance on sentiment analysis, DistilBERT is employed as the base model. Sentiment analysis is a process that extracts subjective information from textual data and categorizes them into different classes. The classification classes may include polarity (positive, neutral, negative) or emotions (happy, sad, angry). In addition, multiple techniques such as fine tuning, regularization and hyperparameter tuning are applied to improve the performance of the model. The proposed solution acquired an accuracy score of 85.41% on Internet Movie Database (IMDB) dataset and 86.59% on Customer Reviews (CR) dataset.

KW - Deep Learning

KW - DistilBERT

KW - Sentiment Analysis

KW - Transformers

UR - https://www.scopus.com/pages/publications/85186676842

U2 - 10.1109/ICSPC59664.2023.10420272

DO - 10.1109/ICSPC59664.2023.10420272

M3 - Conference Paper

AN - SCOPUS:85186676842

SN - 9798350340877

SP - 84

EP - 89

BT - 2023 IEEE 11th Conference on Systems, Process & Control (ICSPC) - Conference Proceedings

A2 - Adnan, Ramli

PB - IEEE, Institute of Electrical and Electronics Engineers

CY - Piscataway NJ USA

T2 - IEEE Conference on Systems, Process and Control (ICSPC) 2023

Y2 - 16 December 2023 through 16 December 2023

ER -

Sentiment Analysis using DistilBERT

Abstract

Conference

Keywords

Access to Document

Other files and links

Cite this