Bugzilla – Bug 3866
Support for parametric or array job types
Last modified: 2012-09-05 11:42:56
You need to log in before you can comment on or make changes to this bug.
No support for parametric jobs. Also known as array jobs. GT4 Currently supports: single, multi, mpi and condor. The enhancement would submit one job which represents N sub-jobs, also known as tasks or steps. This would invoke the same executable with different input parameters, each sub-job is passed a number: 0 to N-1 along with its arguments (future enhancements could extend this to be multiple ranges each with different start, end, and increment/decrement). Each job is not complete until all sub-jobs complete. We have added the new job type “parametric” value in RSL’s job_description.xsd for SimpleType JobTypeEnumeration to allow <jobType>parametric<jobType>. We have used the <count> element to represent the number of sub jobs to be submitted for aggregate job. Note: LSF/Symphony allows a comma-separated index-list of start[-end[:step]] where end defaults to 1000, and step to 1, but we do not need this level of detail initially. We added a stanza in each globus-job-scheduler script to decode job type and then use count to create a parametric job. Note there has been some previous discussion and a bugzilla entries for this such as 3384 and 3569. We have locally modified GT4.0.0 to accomplish this and would like to discuss the potential if this could be integrated back into the main-line code base.
I don't want to create a new bugzilla entry, so I'm attaching this here under this GRAM entry even though it is related to MDS because it logically is coupled. MDS Sub-Job Information from Scheduler Provider. Current: Assume proposed support for parametric jobs (see previously) Each <Job> does not portray sub-<Job> and their states. Enhacement: Extend <Job> schema to include sub-<Job>s. A <Job> should include <TotalJobs>, <RunningJobs>, and <WaitingJobs> for sub-<Jobs>, and a list of sub-<Job>s and their states. Extend each new CE-type Scheduler Provider to include a list of individual sub-<Job>s within a <Job> which is within a <ComputingElement> for each grid scheduler. For example, get list of jobs, for each job check for sub-jobs, if found get list of sub-jobs, and extract results.
Reassigning to current GRAM developer to close/fix as appropriate.
Doing some bugzilla cleanup... Resolving old GRAM3 and GRAM4 issues that are no longer relevant since we've moved on to GRAM5. Also, we're now tracking issue in jira. Any new issues should be added here: http://jira.globus.org/secure/VersionBoard.jspa?selectedProjectId=10363