Abstract
Automatic code summarization generates high-level natural language descriptions of code snippets, which can benefit software maintenance and code comprehension. Recently, Transformer-based models achieved state-of-the-art performance on code summarization tasks. However, there are data gaps in neural model training for some programming languages. To fill this gap, we propose a novel transfer learning approach to accurately transfer knowledge between Transformer-based models. We train a discriminator to identify which heads of the multi-head attention module should be transferred. On this basis, we define a transfer strategy of parameter matrices. We evaluated the proposed transfer learning approach on four state-of-the-art Transformer-based code summarization models. Experimental results show that models with transferred knowledge outperform original models up to 10.70% in BLEU, 5.36% in ROUGE-L, and 4.34% in METEOR.
Original language | English |
---|---|
Title of host publication | SEKE 2022 - Proceedings of the 34th International Conference on Software Engineering and Knowledge Engineering |
Editors | Rong Peng, Carlos Eduardo Pantoja, Pankaj Kamthan |
Place of Publication | Skokie IL USA |
Publisher | Knowledge Systems Institute |
Pages | 91-94 |
Number of pages | 4 |
ISBN (Electronic) | 1891706543, 9781891706547 |
DOIs | |
Publication status | Published - 2022 |
Event | International Conference on Software Engineering and Knowledge Engineering 2022 - Pittsburgh, United States of America Duration: 1 Jul 2022 → 10 Jul 2022 Conference number: 34th http://ksiresearch.org/seke/seke22.html (Website) |
Publication series
Name | Proceedings of the International Conference on Software Engineering and Knowledge Engineering, SEKE |
---|---|
Publisher | Knowledge Systems Institute Graduate School |
ISSN (Print) | 2325-9000 |
ISSN (Electronic) | 2325-9086 |
Conference
Conference | International Conference on Software Engineering and Knowledge Engineering 2022 |
---|---|
Abbreviated title | SEKE 2022 |
Country/Territory | United States of America |
City | Pittsburgh |
Period | 1/07/22 → 10/07/22 |
Internet address |
|
Keywords
- Code Summarization
- Transfer Learning