Grid Harvest Service
|
|
Grid Harvest Service |
|
GHS stands for Grid Harvest Service. It is a performance evaluation and task scheduling system for solving large-scale applications in a shared environment. GHS is based on a novel performance prediction model and a set of task scheduling algorithms. GHS supports three classes of task scheduling, single task, parallel processing and meta-task. The Grid Harvest Service system comprises of five primary subsystems: performance evaluation, performance measurement, task allocation, task scheduling, and execution management. Coordinately, they provide appropriate services to harvest Grid computing.
Conventional performance evaluation and task scheduling systems focus on dedicated distributed systems. Grid computing infrastructure, on another hand, is a shared collaborative environment constructed on virtual organizations. Each organization has its own resource management policy. In a shared environment, a remote task competes with local jobs, or even other remote tasks for resources. The completion time of a task is not only determined by its workload but also the availability of the resources. Some scheduling systems, such as AppLeS, PEGASUS, DAGMAN, CONDOR-G, CHIMERA, and Stork, have been developed recently for distributed shared environments. In these systems, the CPU availability is estimated by Network Weather Service (NWS). However, NWS is designed for short-term predictions (up to 5 minutes) and cannot provide appropriate prediction for long-term applications. To address these problems, we design and develop GHS, a long-term, application-level performance prediction and task scheduling system, for Grid computing. Here long-term means the application requiring hours or more, in contrast to minutes, sequential execution time; and application-level performance means the application turn-around time, in contrast to resource availability.
GHS performance evaluation is based on a new performance model, which is derived from a combination of stochastic analysis and direct-numerical simulation. Unlike other stochastic models, this model individually identifies the effects of machine utilization, computing power, local job service, and task allocation on the completion time of a parallel application. It is theoretically sound and practically feasible.
Utilizing the performance prediction, various partition and scheduling algorithms are developed and adopted in the partition and scheduling component, for single sequential task, single parallel task with a given number of sub-tasks, optimal parallel processing, as well as meta-task composed of a group of independent tasks. A heuristic task scheduling algorithm is proposed to find an acceptable solution with a reasonable cost.
GHS uses an adaptive measurement methodology to monitor resource usage pattern, where the measurement frequency is dynamically updated according to the previous measurement history. This method reduces monitoring overhead considerably.
M. Wu, and X.-H. Sun, "Self-adaptive Task Allocation and Scheduling of Meta-tasks in Non-dedicated Heterogeneous Computing," accepted in a special issue of the International Journal of High Performance Computing and Networking (IJHPCN), 2004.
X.-H. Sun, M. Wu, "GHS, A Performance System of Grid Computing," in the Proc. of the NSF Next Generation Software Program Workshop ( in conjunction with IPDPS05), Denver, Colorado, April., 2005.
A. Eswaradass, X.-H. Sun, and M. Wu, "A Neural Network Based Predictive Mechanism for Available Bandwidth," in the Proc. of the 19th International Parallel and Distributed Processing Symposium (IPDPS05), Denver, Colorado, April., 2005.
M. Wu, and X.-H. Sun, "Memory Conscious Task Partition and Scheduling in Grid Environments," in the Proc. of the 5th IEEE/ACM International Workshop on Grid Computing ( in conjunction with SC 2004), pp. 138-145, Pittsburgh, November 2004.
M. Wu, X.-H. Sun, "A General Self-adaptive Task Scheduling System for Non-dedicated Heterogeneous Computing," in the Proc. of the IEEE International Conference on Cluster Computing, 2003, Hong Kong, December 2003.
X.-H. Sun and M. Wu, "Grid Harvest Service: A System for Long-Term, Application-Level Task Scheduling," in the Proc. of the 2003 IEEE International Parallel and Distributed Processing Symposium (IPDPS 2003), Nice, France, April, 2003.
You can find SCS lab publications here.
Related Links
Following related links are useful for beginner to start research into Grid Harvest Service.
Our Old Project :
Publication and Paper database:
Performance Evaluation :
Task Scheduling :
Distributed Systems :
Middleware Initiative
Back to SCS Home Page