Dynalogue: A transformer-based dialogue system with dynamic attention

Businesses face a range of cyber risks, both external threats and internal vulnerabilities that continue to evolve over time. As cyber attacks continue to increase in complexity and sophistication, more organisations will experience them. For this reason, it is important that organisations seek timely consultancy from cyber professionals so that they can respond to and recover from cyber attacks as quickly as possible. However, huge surges in cyber attacks have long left cyber professionals short of what is required to cover the security needs. This problem is getting worse when an increasing number of people choose to work from home during the pandemic because this situation usually yields extra communication cost. In this paper, we propose to develop a cybersecurity-oriented dialogue system, called Dynalogue1, which can provide consultancy online as a cyber professional. For the first time, Dynalogue provides a promising solution to mitigate the need for cyber professionals via automatically generating problem-targeted conversions to victims of cyber attacks. In spite of many dialogue systems developed in the past, Dynalogue provides a distinct capability of handling long and complicated sentences that are common in cybersecurity-related conversations. It is challenging to have this capability because limited memory in dialogue systems can be hard to accommodate sufficient key information of long sentences. To overcome this challenge, Dynalogue utilises an attention mechanism that dynamically captures key semantics within a sentence instead of using fix window to cut off the sentence. To evaluate Dynalogue, we collect 67K real-world conversations (0.6M utterances) from Bleeping Computer2, which is one of the most popular cybersecurity consultancy websites in the world. The results suggest that Dynalogue outperforms all the existing dialogue systems with 1% ∼9% improvements on all different metrics. We further run Dynalogue on the public dataset WikiHow to validate its compatibility in other domains where conversations are also long and complicated. Dynalogue also outperforms all the other methods with at most 2.4% improvement.