Improving Hadoop Cluster Using Mininet Emulator and Docker
Container
Presented and Publicly Supported
Wednesday April 25, 2018
By Wajdi HAJJI, elhajjiwajdi@gmail.com
Laboratory of Advanced Technology and Intelligent Systems (LATIS), National School of Engineers of Sousse (ENISO), Higher Institute of Computer Science and Management of Kairouan (ISIGK)
Improving Hadoop Cluster Using Mininet Emulator and Docker Container
ABSTRACT:
Large data processing becomes progressively important so that everyone can extract meaningful information from its volume. Faced with this evolution, Apache Hadoop is the main podium of Big Data technology, which mainly serves the intensive data management by ran-king a better performance in a multi-node Hadoop environment.
Our work focuses on the Hadoop application problem, because the two advanced HDFS and Map-Reduce tools in this ecosystem use host machine resources in an intensive way, with a heavy processing time, high memory and processor consumption, and a complexity at the deployment level the Hadoop application on the Cluster.
In this context, the idea is to use the Docker container virtualization technique with the Mininet emulator, to improve the performance of our Hadoop Cluster, by applying the SDN (Software-Defined Networking) tools to optimize network traffic and reduce the congestion problem. Hence our emulation is based on the concept of scalability to maintain the performance of our Hadoop Cluster.
KEYWORDS : Big Data, Hadoop Ecosystem, HDFS, MapReduce, Container Virtualization, Docker, LXC, Mininet Emulator, performance of Hadoop Clusters, SDN software-defined network.
