A self-attentional neural architecture for code completion with multi-task learning

Fang Liu, Ge Li, Bolin Wei, Xin Xia, Zhiyi Fu, Zhi Jin

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review


Code completion, one of the most useful features in the IntegratedDevelopment Environments (IDEs), can accelerate software development by suggesting the libraries, APIs, and method namesin real-time. Recent studies have shown that statistical languagemodels can improve the performance of code completion toolsthrough learning from large-scale software repositories. However,these models suffer from three major drawbacks: a) The hierarchical structural information of the programs is not fully utilizedin the program's representation; b) In programs, the semantic relationships can be very long. Existing recurrent neural networksbased language models are not sufficient to model the long-termdependency. c) Existing approaches perform a specific task in onemodel, which leads to the underuse of the information from relatedtasks. To address these challenges, in this paper, we propose a selfattentional neural architecture for code completion with multi-tasklearning. To utilize the hierarchical structural information of theprograms, we present a novel method that considers the path fromthe predicting node to the root node. To capture the long-termdependency in the input programs, we adopt a self-attentional architecture based network as the base language model. To enable theknowledge sharing between related tasks, we creatively propose aMulti-Task Learning (MTL) framework to learn two related tasks incode completion jointly. Experiments on three real-world datasetsdemonstrate the effectiveness of our model when compared withstate-of-the-art methods.

Original languageEnglish
Title of host publicationProceedings - 2020 IEEE/ACM 28th International Conference on Program Comprehension, ICPC 2020
EditorsYann-Gaël Guéhéneuc, Shinpei Hayashi
Place of PublicationNew York NY USA
PublisherIEEE, Institute of Electrical and Electronics Engineers
Number of pages11
ISBN (Electronic)9781450379588
Publication statusPublished - 2020
EventInternational Conference on Program Comprehension 2020 - Seoul, Korea, Republic of (South)
Duration: 13 Jul 202015 Jul 2020
Conference number: 28th
https://dl.acm.org/doi/proceedings/10.1145/3387904 (Proceedings)
https://conf.researchr.org/home/icpc-2020 (Website)


ConferenceInternational Conference on Program Comprehension 2020
Abbreviated titleICPC 2020
CountryKorea, Republic of (South)
Internet address


  • Code completion
  • Hierarchical structure
  • Multi-task learning
  • Selfattention

Cite this