Abstract
Scheduling tasks in the vicinity of stored data can significantly diminish network traffic. Scheduling optimisation can improve data locality by attempting to locate a task and its related data on the same node. Existing schedulers tend to ignore overhead and tradeoff between data transfer and task placement, and bandwidth consumption, by only emphasising data locality without considering other factors. We present a novel data locality aware scheduler for balancing time consumption and network bandwidth traffic-DLAforBT-to improve data locality for tasks and throughput, with the optimal placement policy exhibiting a threshold-based structure. DLAforBT uses bipartite graph modelling to represent data placement, adopts a judgment mechanism and a precise prediction model to determine moving data or moving computation. It integrates an improved Dominant Resource Fairness (DRF) resource allocation to capture tenants' resource allocation and run as many jobs as possible. DLAforBT improves by 16% of data locality rate, and 25% of throughput.
| Original language | English |
|---|---|
| Title of host publication | Proceedings - 2019 IEEE International Conference on Cloud Computing - IEEE CLOUD 2019 - Part of the 2019 IEEE World Congress on Services |
| Editors | Elisa Bertino, Carl K. Chang, Peter Chen, Ernesto Damiani, Michael Goul, Katsunori Oyama |
| Place of Publication | Piscataway NJ USA |
| Publisher | IEEE, Institute of Electrical and Electronics Engineers |
| Pages | 496-498 |
| Number of pages | 3 |
| ISBN (Electronic) | 9781728127057, 9781728127040 |
| ISBN (Print) | 9781728127064 |
| DOIs | |
| Publication status | Published - 2019 |
| Event | International Conference on Cloud Computing 2019 - Milan, Italy Duration: 8 Jul 2019 → 13 Jul 2019 Conference number: 12th https://conferences.computer.org/cloud/2019/ |
Conference
| Conference | International Conference on Cloud Computing 2019 |
|---|---|
| Abbreviated title | CLOUD 2019 |
| Country | Italy |
| City | Milan |
| Period | 8/07/19 → 13/07/19 |
| Internet address |
Keywords
- Bipar tite graph modelling
- Cloud computing
- Data locality
- Multi-tenancy
- Scheduling
Cite this
}
A highly efficient data locality aware task scheduler for cloud-based systems. / Jia, Ru; Yang, Yun; Grundy, John; Keung, Jacky; Li, Hao.
Proceedings - 2019 IEEE International Conference on Cloud Computing - IEEE CLOUD 2019 - Part of the 2019 IEEE World Congress on Services. ed. / Elisa Bertino; Carl K. Chang; Peter Chen; Ernesto Damiani; Michael Goul; Katsunori Oyama. Piscataway NJ USA : IEEE, Institute of Electrical and Electronics Engineers, 2019. p. 496-498 8814565.Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review
TY - GEN
T1 - A highly efficient data locality aware task scheduler for cloud-based systems
AU - Jia, Ru
AU - Yang, Yun
AU - Grundy, John
AU - Keung, Jacky
AU - Li, Hao
PY - 2019
Y1 - 2019
N2 - Scheduling tasks in the vicinity of stored data can significantly diminish network traffic. Scheduling optimisation can improve data locality by attempting to locate a task and its related data on the same node. Existing schedulers tend to ignore overhead and tradeoff between data transfer and task placement, and bandwidth consumption, by only emphasising data locality without considering other factors. We present a novel data locality aware scheduler for balancing time consumption and network bandwidth traffic-DLAforBT-to improve data locality for tasks and throughput, with the optimal placement policy exhibiting a threshold-based structure. DLAforBT uses bipartite graph modelling to represent data placement, adopts a judgment mechanism and a precise prediction model to determine moving data or moving computation. It integrates an improved Dominant Resource Fairness (DRF) resource allocation to capture tenants' resource allocation and run as many jobs as possible. DLAforBT improves by 16% of data locality rate, and 25% of throughput.
AB - Scheduling tasks in the vicinity of stored data can significantly diminish network traffic. Scheduling optimisation can improve data locality by attempting to locate a task and its related data on the same node. Existing schedulers tend to ignore overhead and tradeoff between data transfer and task placement, and bandwidth consumption, by only emphasising data locality without considering other factors. We present a novel data locality aware scheduler for balancing time consumption and network bandwidth traffic-DLAforBT-to improve data locality for tasks and throughput, with the optimal placement policy exhibiting a threshold-based structure. DLAforBT uses bipartite graph modelling to represent data placement, adopts a judgment mechanism and a precise prediction model to determine moving data or moving computation. It integrates an improved Dominant Resource Fairness (DRF) resource allocation to capture tenants' resource allocation and run as many jobs as possible. DLAforBT improves by 16% of data locality rate, and 25% of throughput.
KW - Bipar tite graph modelling
KW - Cloud computing
KW - Data locality
KW - Multi-tenancy
KW - Scheduling
UR - http://www.scopus.com/inward/record.url?scp=85072308889&partnerID=8YFLogxK
U2 - 10.1109/CLOUD.2019.00089
DO - 10.1109/CLOUD.2019.00089
M3 - Conference Paper
SN - 9781728127064
SP - 496
EP - 498
BT - Proceedings - 2019 IEEE International Conference on Cloud Computing - IEEE CLOUD 2019 - Part of the 2019 IEEE World Congress on Services
A2 - Bertino, Elisa
A2 - Chang, Carl K.
A2 - Chen, Peter
A2 - Damiani, Ernesto
A2 - Goul, Michael
A2 - Oyama, Katsunori
PB - IEEE, Institute of Electrical and Electronics Engineers
CY - Piscataway NJ USA
ER -