Bugzilla – Bug 3866
Support for parametric or array job types
Last modified: 2012-09-05 11:42:56
You need to
before you can comment on or make changes to this bug.
No support for parametric jobs. Also known as array jobs. GT4 Currently
supports: single, multi, mpi and condor. The enhancement would submit one job
which represents N sub-jobs, also known as tasks or steps. This would invoke the
same executable with different input parameters, each sub-job is passed a
number: 0 to N-1 along with its arguments (future enhancements could extend this
to be multiple ranges each with different start, end, and increment/decrement).
Each job is not complete until all sub-jobs complete.
We have added the new job type “parametric” value in RSL’s job_description.xsd
for SimpleType JobTypeEnumeration to allow <jobType>parametric<jobType>. We have
used the <count> element to represent the number of sub jobs to be submitted for
aggregate job. Note: LSF/Symphony allows a comma-separated index-list of
start[-end[:step]] where end defaults to 1000, and step to 1, but we do not need
this level of detail initially. We added a stanza in each globus-job-scheduler
script to decode job type and then use count to create a parametric job.
Note there has been some previous discussion and a bugzilla entries for this
such as 3384 and 3569.
We have locally modified GT4.0.0 to accomplish this and would like to discuss
the potential if this could be integrated back into the main-line code base.
I don't want to create a new bugzilla entry, so I'm attaching this here under
this GRAM entry even though it is related to MDS because it logically is coupled.
MDS Sub-Job Information from Scheduler Provider.
Current: Assume proposed support for parametric jobs (see previously)
Each <Job> does not portray sub-<Job> and their states.
Enhacement: Extend <Job> schema to include sub-<Job>s. A <Job> should include
<TotalJobs>, <RunningJobs>, and <WaitingJobs> for sub-<Jobs>, and a list of
sub-<Job>s and their states. Extend each new CE-type Scheduler Provider to
include a list of individual sub-<Job>s within a <Job> which is within a
<ComputingElement> for each grid scheduler. For example, get list of jobs, for
each job check for sub-jobs, if found get list of sub-jobs, and extract results.
Reassigning to current GRAM developer to close/fix as appropriate.
Doing some bugzilla cleanup... Resolving old GRAM3 and GRAM4 issues that are
no longer relevant since we've moved on to GRAM5. Also, we're now tracking
issue in jira. Any new issues should be added here: