Sentiment analysis for telco popularity on twitter big data using a novel Malaysian dictionary

Malaysians are actively expressing feelings and opinions on social networks such as Twitter. These expressions can be harvested for studying the customer sentiments towards certain brands and preferences of customers. As business analytics becoming more important, sentiment analysis may provide crucial information in making customer-driven decisions. Therefore, accuracy is critical in determining the reliability and integrity of the analysis. Although, processing massive messages on social media is a huge challenge, it is now made easier by the advancement of the big data architecture. There are many techniques in interpreting these messages. However, Malaysians consists of people from very diversified backgrounds with a multitude of cultures and languages in daily use. Therefore, it is very common to find messages on social media with mixture of various local languages and slangs. The slangs used are mostly dialects expressed with alphabets. This project explores the techniques on analyzing the popularity of 5 telecommunication companies in Malaysia and addresses the shortfalls of using the existing English sentiment dictionary. With the accomplishment of this project, a new localized dictionary is developed by compiling various mixtures of English, Malay localized sentiwords and slangs into the dictionary. The new dictionary is proven to capture and analyze 30% extra keywords on the Malaysian tweets sampled. These additional matches will improve the accuracy compared to existing dictionaries.