Ajou University repository

EPPADS: An enhanced phase-based performance-aware dynamic scheduler for high job execution performance in large scale clusters
Citations

SCOPUS

0

Citation Export

Publication Year
2019-01-01
Journal
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Publisher
Springer Verlag
Citation
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol.11446 LNCS, pp.140-156
Keyword
Distributed processingMapReduceScheduling
Mesh Keyword
Communication overheadsDistributed processingLarge-scale clustersMap-reduceMapreduce frameworksOptimal performanceScheduling mechanismSpeculative execution
All Science Classification Codes (ASJC)
Theoretical Computer ScienceComputer Science (all)
Abstract
The way in which jobs are scheduled is critical to achieve high job processing performance in large scale data clusters. Most existing scheduling mechanism employs a First-In First-Out, serialized approach encompassed with task straggler hunting techniques which launches speculative tasks after detecting slow tasks. This is often achieved through the instrumentation of processing nodes. Such node instrumentation incurs frequent communication overheads as the number of processing nodes increase. Moreover the sequential scheduling of job tasks and the straggler hunting approach fails to meet optimal performance as they increase job waiting time in queue and incurs delayed speculative execution of straggling tasks respectively. In this paper we propose an Enhanced Phase based Performance Aware Dynamic Scheduler (EPPADS), which schedules job tasks without additional instrumentation modules. EPPADS uses a two staged scheduling approach, that is, the slow start phase (SSP) and accelerate phase (AccP). The SSP schedules the initial task in the queue in the normal FIFO way and records the initial execution times of the processing nodes. The AccP uses the initial execution times to compute the processing nodes task distribution ratio of the remaining tasks and schedules them using a single scheduling I/O. We implement EPPADS scheduler in Hadoop’s MapReduce framework. Our evaluation shows that EPPADS can achieve a performance improvement on FIFO scheduler of 30%. Compared with existing Dynamic scheduling approach which uses node instrumentation, EPPADS achieves a better performance of 22%.
Language
eng
URI
https://aurora.ajou.ac.kr/handle/2018.oak/36407
https://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=85065551780&origin=inward
DOI
https://doi.org/10.1007/978-3-030-18576-3_9
Journal URL
https://www.springer.com/series/558
Type
Conference
Funding
Acknowledgement. This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2016R1D1A1B03934129).
Show full item record

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.

Related Researcher

HAMANDAWANA PRINCE Image
HAMANDAWANA PRINCEHAMANDAWANA, PRINCE
Department of Software and Computer Engineering
Read More

Total Views & Downloads

File Download

  • There are no files associated with this item.