Ajou University repository

Novel data-placement scheme for improving the data locality of Hadoop in heterogeneous environments
Citations

SCOPUS

11

Citation Export

Publication Year
2021-09-25
Publisher
John Wiley and Sons Ltd
Citation
Concurrency and Computation: Practice and Experience, Vol.33
Keyword
data localitydata placementHadoop MapReduceheterogeneous environmentreplication
Mesh Keyword
Data localityData placementHadoop MapReduceHeterogeneous environmentsreplication
All Science Classification Codes (ASJC)
SoftwareTheoretical Computer ScienceComputer Science ApplicationsComputer Networks and CommunicationsComputational Theory and Mathematics
Abstract
To address the challenging needs of high-performance big data processing, parallel-distributed frameworks such as Hadoop are being utilized extensively. However, in heterogeneous environments, the performance of Hadoop clusters is below par. This is primarily because the blocks of the clusters are allocated equally to all nodes without regard to differences in the capability of individual nodes. This results in reduced data locality. Thus, a new data-placement scheme that enhances data locality is required for Hadoop in heterogeneous environments. This article proposes a new data placement scheme that preserves the same degree of data locality in heterogeneous environments as that of the standard Hadoop, with only a small amount of replicated data. In the proposed scheme, only those blocks with the highest probability of being accessed remotely are selected and replicated. The results of experiments conducted indicate that the proposed scheme incurs only a 20% disk space overhead and has virtually the same data locality ratio as the standard Hadoop, which has a replication factor of three and 200% disk space overhead.
Language
eng
URI
https://dspace.ajou.ac.kr/dev/handle/2018.oak/31246
DOI
https://doi.org/10.1002/cpe.5752
Fulltext

Type
Conference Paper
Funding
This research is supported by C2 integrating and interfacing technologies laboratory of Agency for Defense Development (UD180010ED).
Show full item record

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.

Related Researcher

Oh, Sangyoon Image
Oh, Sangyoon오상윤
Department of Software and Computer Engineering
Read More

Total Views & Downloads

File Download

  • There are no files associated with this item.