Service interruption on Monday 11 July from 12:30 to 13:00: all the sites of the CCSD (HAL, EpiSciences, SciencesConf, AureHAL) will be inaccessible (network hardware connection).
Skip to Main content Skip to Navigation
Master Thesis

Immediate data management for map/reduce applications

Abstract : Map/Reduce is a popular programming model and an associated implementation for processing large data sets nowadays. This report aims to present the problem of managing intermediate data which is generated during Map/Reduce computations. We focus on the Hadoop Map/Reduce framework and two file systems, Hadoop Distributed File System and BlobSeer File System, used as storage backends of Hadoop Map/Reduce framework, which substitute for the original intermediate storage layer. As a result, our new design deals with data reliability and efficiency, thanks to BlobSeer which supports access concurrency. The prototype has been experimented on the Grid'5000 testbed, using up to 150 nodes.
Complete list of metadata

Cited literature [19 references]  Display  Hide  Download
Contributor : Co-Responsables Du Mri V. Gouranton Et S. Blazy Connect in order to contact the contributor
Submitted on : Friday, October 29, 2010 - 7:48:27 PM
Last modification on : Monday, February 15, 2021 - 10:37:19 AM
Long-term archiving on: : Sunday, January 30, 2011 - 3:08:26 AM


  • HAL Id : dumas-00530784, version 1


Thi-Thu-Lan Trieu. Immediate data management for map/reduce applications. Calcul parallèle, distribué et partagé [cs.DC]. 2010. ⟨dumas-00530784⟩



Record views


Files downloads