Self-adaptive attribute weighting for Naive Bayes classification

Jia Wu, Shirui Pan, Xingquan Zhu, Zhihua Cai, Peng Zhang, Chengqi Zhang

Research output: Contribution to journalArticleResearchpeer-review

Abstract

Naive Bayes (NB) is a popular machine learning tool for classification, due to its simplicity, high computational efficiency, and good classification accuracy, especially for high dimensional data such as texts. In reality, the pronounced advantage of NB is often challenged by the strong conditional independence assumption between attributes, which may deteriorate the classification performance. Accordingly, numerous efforts have been made to improve NB, by using approaches such as structure extension, attribute selection, attribute weighting, instance weighting, local learning and so on. In this paper, we propose a new Artificial Immune System (AIS) based self-adaptive attribute weighting method for Naive Bayes classification. The proposed method, namely AISWNB, uses immunity theory in Artificial Immune Systems to search optimal attribute weight values, where self-adjusted weight values will alleviate the conditional independence assumption and help calculate the conditional probability in an accurate way. One noticeable advantage of AISWNB is that the unique immune system based evolutionary computation process, including initialization, clone, section, and mutation, ensures that AISWNB can adjust itself to the data without explicit specification of functional or distributional forms of the underlying model. As a result, AISWNB can obtain good attribute weight values during the learning process. Experiments and comparisons on 36 machine learning benchmark data sets and six image classification data sets demonstrate that AISWNB significantly outperforms its peers in classification accuracy, class probability estimation, and class ranking performance.

Original languageEnglish
Pages (from-to)1487-1502
Number of pages16
JournalExpert Systems with Applications
Volume42
Issue number3
DOIs
Publication statusPublished - 15 Feb 2015
Externally publishedYes

Keywords

  • Artificial Immune Systems
  • Attribute weighting
  • Evolutionary computing
  • Naive Bayes Self-adaptive

Cite this

Wu, Jia ; Pan, Shirui ; Zhu, Xingquan ; Cai, Zhihua ; Zhang, Peng ; Zhang, Chengqi. / Self-adaptive attribute weighting for Naive Bayes classification. In: Expert Systems with Applications. 2015 ; Vol. 42, No. 3. pp. 1487-1502.
@article{bc12b7fa1daf4661926b7911d0db1f2d,
title = "Self-adaptive attribute weighting for Naive Bayes classification",
abstract = "Naive Bayes (NB) is a popular machine learning tool for classification, due to its simplicity, high computational efficiency, and good classification accuracy, especially for high dimensional data such as texts. In reality, the pronounced advantage of NB is often challenged by the strong conditional independence assumption between attributes, which may deteriorate the classification performance. Accordingly, numerous efforts have been made to improve NB, by using approaches such as structure extension, attribute selection, attribute weighting, instance weighting, local learning and so on. In this paper, we propose a new Artificial Immune System (AIS) based self-adaptive attribute weighting method for Naive Bayes classification. The proposed method, namely AISWNB, uses immunity theory in Artificial Immune Systems to search optimal attribute weight values, where self-adjusted weight values will alleviate the conditional independence assumption and help calculate the conditional probability in an accurate way. One noticeable advantage of AISWNB is that the unique immune system based evolutionary computation process, including initialization, clone, section, and mutation, ensures that AISWNB can adjust itself to the data without explicit specification of functional or distributional forms of the underlying model. As a result, AISWNB can obtain good attribute weight values during the learning process. Experiments and comparisons on 36 machine learning benchmark data sets and six image classification data sets demonstrate that AISWNB significantly outperforms its peers in classification accuracy, class probability estimation, and class ranking performance.",
keywords = "Artificial Immune Systems, Attribute weighting, Evolutionary computing, Naive Bayes Self-adaptive",
author = "Jia Wu and Shirui Pan and Xingquan Zhu and Zhihua Cai and Peng Zhang and Chengqi Zhang",
year = "2015",
month = "2",
day = "15",
doi = "10.1016/j.eswa.2014.09.019",
language = "English",
volume = "42",
pages = "1487--1502",
journal = "Expert Systems with Applications",
issn = "0957-4174",
publisher = "Elsevier",
number = "3",

}

Self-adaptive attribute weighting for Naive Bayes classification. / Wu, Jia; Pan, Shirui; Zhu, Xingquan; Cai, Zhihua; Zhang, Peng; Zhang, Chengqi.

In: Expert Systems with Applications, Vol. 42, No. 3, 15.02.2015, p. 1487-1502.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - Self-adaptive attribute weighting for Naive Bayes classification

AU - Wu, Jia

AU - Pan, Shirui

AU - Zhu, Xingquan

AU - Cai, Zhihua

AU - Zhang, Peng

AU - Zhang, Chengqi

PY - 2015/2/15

Y1 - 2015/2/15

N2 - Naive Bayes (NB) is a popular machine learning tool for classification, due to its simplicity, high computational efficiency, and good classification accuracy, especially for high dimensional data such as texts. In reality, the pronounced advantage of NB is often challenged by the strong conditional independence assumption between attributes, which may deteriorate the classification performance. Accordingly, numerous efforts have been made to improve NB, by using approaches such as structure extension, attribute selection, attribute weighting, instance weighting, local learning and so on. In this paper, we propose a new Artificial Immune System (AIS) based self-adaptive attribute weighting method for Naive Bayes classification. The proposed method, namely AISWNB, uses immunity theory in Artificial Immune Systems to search optimal attribute weight values, where self-adjusted weight values will alleviate the conditional independence assumption and help calculate the conditional probability in an accurate way. One noticeable advantage of AISWNB is that the unique immune system based evolutionary computation process, including initialization, clone, section, and mutation, ensures that AISWNB can adjust itself to the data without explicit specification of functional or distributional forms of the underlying model. As a result, AISWNB can obtain good attribute weight values during the learning process. Experiments and comparisons on 36 machine learning benchmark data sets and six image classification data sets demonstrate that AISWNB significantly outperforms its peers in classification accuracy, class probability estimation, and class ranking performance.

AB - Naive Bayes (NB) is a popular machine learning tool for classification, due to its simplicity, high computational efficiency, and good classification accuracy, especially for high dimensional data such as texts. In reality, the pronounced advantage of NB is often challenged by the strong conditional independence assumption between attributes, which may deteriorate the classification performance. Accordingly, numerous efforts have been made to improve NB, by using approaches such as structure extension, attribute selection, attribute weighting, instance weighting, local learning and so on. In this paper, we propose a new Artificial Immune System (AIS) based self-adaptive attribute weighting method for Naive Bayes classification. The proposed method, namely AISWNB, uses immunity theory in Artificial Immune Systems to search optimal attribute weight values, where self-adjusted weight values will alleviate the conditional independence assumption and help calculate the conditional probability in an accurate way. One noticeable advantage of AISWNB is that the unique immune system based evolutionary computation process, including initialization, clone, section, and mutation, ensures that AISWNB can adjust itself to the data without explicit specification of functional or distributional forms of the underlying model. As a result, AISWNB can obtain good attribute weight values during the learning process. Experiments and comparisons on 36 machine learning benchmark data sets and six image classification data sets demonstrate that AISWNB significantly outperforms its peers in classification accuracy, class probability estimation, and class ranking performance.

KW - Artificial Immune Systems

KW - Attribute weighting

KW - Evolutionary computing

KW - Naive Bayes Self-adaptive

UR - http://www.scopus.com/inward/record.url?scp=84908426170&partnerID=8YFLogxK

U2 - 10.1016/j.eswa.2014.09.019

DO - 10.1016/j.eswa.2014.09.019

M3 - Article

VL - 42

SP - 1487

EP - 1502

JO - Expert Systems with Applications

JF - Expert Systems with Applications

SN - 0957-4174

IS - 3

ER -