Projects per year
Abstract
One significant obstacle to the successful application of machine learning to real-world data is that of labeling: it is often prohibitively expensive to pay an ethical amount for the human labor required to label a dataset successfully. Human-in-the-loop techniques such as active learning can reduce the cost, but the required human time is still significant and many fixed costs remain. Another option is to employ pre-trained transformer models as labelers at scale, which can yield reasonable accuracy and significant cost savings. However, such models can still be expensive to use due to their high computational requirements, and the opaque nature of these models is not always suitable in applied social science and public use contexts. We propose a novel semi-supervised method, named Slingshot Learning, in which we iteratively and selectively augment a small human-labeled dataset with labels from a high-quality "teacher" model to slingshot the performance of a "student" model in a cost-efficient manner. This reduces the accuracy trade-off required to use these simpler algorithms without disrupting their benefits, such as lower compute requirements, better interpretability, and faster inference. We define and discuss the slingshot learning algorithm and demonstrate its effectiveness on several benchmark tasks, using ALBERT to teach a simple Naive Bayes binary classifier. We experimentally demonstrate that Slingshot learning effectively decreases the performance gap between the teacher and student models. We also analyze its performance in several scenarios and compare different variants of the algorithm.
Original language | English |
---|---|
Title of host publication | EACL 2023 |
Subtitle of host publication | the 17th Conference of the European Chapter of the Association for Computational Linguistics, Proceedings of the Conference |
Editors | Isabelle Augenstein, Andreas Vlachos |
Place of Publication | Stroudsburg PA USA |
Publisher | Association for Computational Linguistics (ACL) |
Pages | 3233-3247 |
Number of pages | 15 |
ISBN (Electronic) | 9781959429449 |
Publication status | Published - 2023 |
Event | European Association of Computational Linguistics Conference 2023 - Dubrovnik, Croatia Duration: 2 May 2023 → 6 May 2023 Conference number: 17th https://2023.eacl.org/ (Website) https://aclanthology.org/volumes/2023.eacl-main/ (Proceedings) |
Conference
Conference | European Association of Computational Linguistics Conference 2023 |
---|---|
Abbreviated title | EACL 2023 |
Country/Territory | Croatia |
City | Dubrovnik |
Period | 2/05/23 → 6/05/23 |
Internet address |
|
Projects
- 1 Finished
-
Understanding policy, media, and academic narratives around cycles of disadvantage in Australia
Faulkner, N., Dwyer, T., Smith, L., Bragge, P., Angus, S., Raschky, P., Buntine, W., Webb, G., Batstone, J. & Goodwin, S.
11/01/21 → 26/02/22
Project: Research