Leveraging latent economic concepts and sentiments in the news for market prediction

Saeede Anbaee Farimani, Majid Vafaei Jahan, Amin Milani Fard, Reza Haffari

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review


Most of the existing news-based market prediction techniques disregard conceptual and emotional relations in the news stream. In this work, we consider the conceptual relationship between news documents using contextualized latent concept modeling as well as leveraging news sentiment and technical indicators. We present our approach as an open-source RESTFul API. We build a corpus of financial news related to currency pairs in the Foreign Exchange and Cryptocurrencies markets. Next, we apply BERT-based embedding to generate word vectors, cluster the vectors to create latent economic concepts, and propose a document representation based on the distribution of words on these concepts as well as news sentiment. We use a recurrent convolutional neural network to jointly use BERT-based text representation and technical indicators embedding for market time series prediction. We further augment our model with technical indicators using another recurrent layer. The experimental results show the superiority of our method compared to the baselines. Our MarketNews dataset, news crawler, and MarketPredict APIs are available for public use.
Original languageEnglish
Title of host publication2021 IEEE 8th International Conference on Data Science and Advanced Analytics (DSAA 2021)
EditorsPedro Larrañaga, Albert Bifet
Place of PublicationPiscataway NJ USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages10
ISBN (Electronic)9781665420990
ISBN (Print)9781665421003
Publication statusPublished - 2021
EventIEEE International Conference on Data Science and Advanced Analytics 2021 - Porto, Portugal
Duration: 6 Oct 20219 Oct 2021
Conference number: 8th
https://dsaa2021.dcc.fc.up.pt/organization/organizing-committee (Website)


ConferenceIEEE International Conference on Data Science and Advanced Analytics 2021
Abbreviated titleDSAA 2021
Internet address

Cite this