Immediate data management for map/reduce applications - DUMAS - Dépôt Universitaire de Mémoires Après Soutenance
Master Thesis Year : 2010

Immediate data management for map/reduce applications

Abstract

Map/Reduce is a popular programming model and an associated implementation for processing large data sets nowadays. This report aims to present the problem of managing intermediate data which is generated during Map/Reduce computations. We focus on the Hadoop Map/Reduce framework and two file systems, Hadoop Distributed File System and BlobSeer File System, used as storage backends of Hadoop Map/Reduce framework, which substitute for the original intermediate storage layer. As a result, our new design deals with data reliability and efficiency, thanks to BlobSeer which supports access concurrency. The prototype has been experimented on the Grid'5000 testbed, using up to 150 nodes.

Domains

Distributed, Parallel, and Cluster Computing [cs.DC] Operating Systems [cs.OS]
Fichier principal
Vignette du fichier
Trieu_Thi_Thu_Lan.pdf (1.07 Mo) Télécharger le fichier

Co-Responsables Du Mri V. Gouranton Et S. Blazy  : Connect in order to contact the contributor

https://dumas.ccsd.cnrs.fr/dumas-00530784

Submitted on : Friday, October 29, 2010-7:48:27 PM

Last modification on : Monday, February 15, 2021-10:37:19 AM

Long-term archiving on : Sunday, January 30, 2011-3:08:26 AM

Download

Dates and versions

dumas-00530784 , version 1 (29-10-2010)

Identifiers

  • HAL Id : dumas-00530784 , version 1

Cite

Thi-Thu-Lan Trieu. Immediate data management for map/reduce applications. Calcul parallèle, distribué et partagé [cs.DC]. 2010. ⟨dumas-00530784⟩

Export

BibTeX XML-TEI Dublin Core DC Terms EndNote DataCite

Collections

ENS-CACHAN GRID5000 MEM-MRI DUMAS SILECS ENS-PARIS-SACLAY
108 View
261 Download

Share

More