TY - JOUR
T1 - Dependable large scale behavioral patterns mining from sensor data using Hadoop platform
AU - Rashid, Md. Mamunur
AU - Gondal, Iqbal
AU - Kamruzzaman, Joarder
PY - 2017/2/10
Y1 - 2017/2/10
N2 - Wireless sensor networks (WSNs) will be an integral part of the future Internet of Things (IoT) environment and generate large volumes of data. However, these data would only be of benefit if useful knowledge can be mined from them. A data mining framework for WSNs includes data extraction, storage and mining techniques, and must be efficient and dependable. In this paper, we propose a new type of behavioral pattern mining technique from sensor data called regularly frequent sensor patterns (RFSPs). RFSPs can identify a set of temporally correlated sensors which can reveal significant knowledge from the monitored data. A distributed data extraction model to prepare the data required for mining RFSPs is proposed, as the distributed scheme ensures higher availability through greater redundancy. The tree structure for RFSP is compact requires less memory and can be constructed using only a single scan through the dataset, and the mining technique is efficient with low runtime. Current mining techniques in the literature on sensor data employ a single memory-based sequential approach and hence are not efficient. Moreover, usage of the MapReduce model for the distributed solution has not been explored extensively. Since MapReduce is becoming the de facto model for computation on large data, we also propose a parallel implementation of the RFSP mining algorithm, called RFSP on Hadoop (RFSP-H), which uses a MapReduce-based framework to gain further efficiency. Experiments conducted to evaluate the compactness and performance of the data extraction model, RFSP-tree and RFSP-H mining show improved results.
AB - Wireless sensor networks (WSNs) will be an integral part of the future Internet of Things (IoT) environment and generate large volumes of data. However, these data would only be of benefit if useful knowledge can be mined from them. A data mining framework for WSNs includes data extraction, storage and mining techniques, and must be efficient and dependable. In this paper, we propose a new type of behavioral pattern mining technique from sensor data called regularly frequent sensor patterns (RFSPs). RFSPs can identify a set of temporally correlated sensors which can reveal significant knowledge from the monitored data. A distributed data extraction model to prepare the data required for mining RFSPs is proposed, as the distributed scheme ensures higher availability through greater redundancy. The tree structure for RFSP is compact requires less memory and can be constructed using only a single scan through the dataset, and the mining technique is efficient with low runtime. Current mining techniques in the literature on sensor data employ a single memory-based sequential approach and hence are not efficient. Moreover, usage of the MapReduce model for the distributed solution has not been explored extensively. Since MapReduce is becoming the de facto model for computation on large data, we also propose a parallel implementation of the RFSP mining algorithm, called RFSP on Hadoop (RFSP-H), which uses a MapReduce-based framework to gain further efficiency. Experiments conducted to evaluate the compactness and performance of the data extraction model, RFSP-tree and RFSP-H mining show improved results.
KW - Data mining
KW - Frequent pattern
KW - Knowledge discovery
KW - MapReduce
KW - Regularly frequent sensor pattern
KW - Wireless sensor networks
UR - http://www.scopus.com/inward/record.url?scp=84977126839&partnerID=8YFLogxK
U2 - 10.1016/j.ins.2016.06.036
DO - 10.1016/j.ins.2016.06.036
M3 - Article
AN - SCOPUS:84977126839
SN - 0020-0255
VL - 379
SP - 128
EP - 145
JO - Information Sciences
JF - Information Sciences
ER -