Skip to Main content Skip to Navigation
Master Thesis

Immediate data management for map/reduce applications

Abstract : Map/Reduce is a popular programming model and an associated implementation for processing large data sets nowadays. This report aims to present the problem of managing intermediate data which is generated during Map/Reduce computations. We focus on the Hadoop Map/Reduce framework and two file systems, Hadoop Distributed File System and BlobSeer File System, used as storage backends of Hadoop Map/Reduce framework, which substitute for the original intermediate storage layer. As a result, our new design deals with data reliability and efficiency, thanks to BlobSeer which supports access concurrency. The prototype has been experimented on the Grid'5000 testbed, using up to 150 nodes.
Complete list of metadata

Cited literature [19 references]  Display  Hide  Download

https://dumas.ccsd.cnrs.fr/dumas-00530784
Contributor : Co-Responsables Du Mri V. Gouranton Et S. Blazy <>
Submitted on : Friday, October 29, 2010 - 7:48:27 PM
Last modification on : Monday, February 15, 2021 - 10:37:19 AM
Long-term archiving on: : Sunday, January 30, 2011 - 3:08:26 AM

Identifiers

  • HAL Id : dumas-00530784, version 1

Citation

Thi-Thu-Lan Trieu. Immediate data management for map/reduce applications. Calcul parallèle, distribué et partagé [cs.DC]. 2010. ⟨dumas-00530784⟩

Share

Metrics

Record views

172

Files downloads

417