A self-attentional neural architecture for code completion with multi-task learning

Code completion, one of the most useful features in the IntegratedDevelopment Environments (IDEs), can accelerate software development by suggesting the libraries, APIs, and method namesin real-time. Recent studies have shown that statistical languagemodels can improve the performance of code completion toolsthrough learning from large-scale software repositories. However,these models suffer from three major drawbacks: a) The hierarchical structural information of the programs is not fully utilizedin the program's representation; b) In programs, the semantic relationships can be very long. Existing recurrent neural networksbased language models are not sufficient to model the long-termdependency. c) Existing approaches perform a specific task in onemodel, which leads to the underuse of the information from relatedtasks. To address these challenges, in this paper, we propose a selfattentional neural architecture for code completion with multi-tasklearning. To utilize the hierarchical structural information of theprograms, we present a novel method that considers the path fromthe predicting node to the root node. To capture the long-termdependency in the input programs, we adopt a self-attentional architecture based network as the base language model. To enable theknowledge sharing between related tasks, we creatively propose aMulti-Task Learning (MTL) framework to learn two related tasks incode completion jointly. Experiments on three real-world datasetsdemonstrate the effectiveness of our model when compared withstate-of-the-art methods.