Managing Long Running Queries in Grid Environment

摘要

Exceptionally large amounts of both distributed data and computational resources are becoming available through the Grid. This will enable efficient exchange and processing of very large amounts of data combined with CPU intensive computations, as required by many scientific applications. We propose a customizable Grid-based query processor built on top of an established Grid infrastructure, NorduGrid. It allows users to submit queries wrapping user-defined long running operations that filter and transform distributed customized data. Limitations imposed by the used Grid infrastructure influence the architecture. For example, resource requirements have to be specified before Grid jobs are started and delays may occur based on the availability of required resources for a job. We are developing a fully functional prototype to investigate the viability of the approach and its applicability. Our first application area is Particle Physics where scientists analyze huge amount of data produced by a collider or simulators to identify particles.