EU DataGrid Project
and Fabric Management

Background: Distributed Processing of Large Data Volumes in High Energy Physics

The Large Hadron Collider (LHC) experiments at Cern in Geneva will produce data, that will be stored distributed all over Europe for offline analysis. Each of these experiments is expected to produce several petabytes of data per year.

Computing in Parallel
- not Parallel Computing

The DataGrid aims at developing a grid to provide online access to data on a petabyte scale. This data is distributed and replicated in a hierarchy of centers with different capabilities. About 6000 scientists will submit their computing jobs to these centers.

Software Components of Fabric Management

From Traditional Tape Storage to Online Data Access

Data density for hard drives doubled:

since 1997: every 12 months

this allows clusters to provide online data storage of some 100TB within the next few years.

	For very large data sets a global file system is needed that distributes one file over multiple IDE disks.
	Data that is replicated can be stored in an unsafe area of the disk.

Network RAID for Online Data Storage

References:

A. Reinefeld, V. Lindenstruth. How to Build a High-Performance Compute Cluster for the Grid. MSA2001, IEEE Computer Society Press.

More Information on the EU DataGrid Project

General

The EU DataGrid project 'Research and Technological Development for an International Data Grid' is a research and development project funded by the 5th IST Framework Programme of the European Commission. It started in January 2001 and runs for three years until December 2003.

The project consortium consists of the coordinator CERN, the partners CNRS, ESRIN, INFN, NIKHEF, PPARC and the associated partners CEA (France), CESNET (Czech Republic), Compagnie des Signaux (FRANCE), Computer and Automation Research Institute (Hungary), CNR (Italy), DATAMAT (Italy), Helsinki Institute of Physics (Finland), IBM (UK), IFAE (Spain), IRST (Italy), KNMI (Netherlands), NFR (Sweden), Ruprecht-Karls-Universität Heidelberg (Germany), SARA (Netherlands), and ZIB (Germany).

WP1:   Grid Workload Management
WP2:   Grid Data Management
WP3:   Grid Monitoring Services
WP4:   Fabric Management
WP5:   Mass Storage Management
WP6:   Integration Testbed
WP7:   Network Services
WP8:   HEP Applications
WP9:   EO Science Applications
WP10: Biology Applications
WP11: Dissemination
WP12: Project Management

Links