Bug 3242 - Software selection thru WS GRAM RSL
: Software selection thru WS GRAM RSL
wsrf managed job factory service
: development
: All All
: P1 enhancement
: 4.4
Assigned To:
  Show dependency treegraph
Reported: 2005-04-28 16:54 by
Modified: 2012-09-05 11:42 (History)



You need to log in before you can comment on or make changes to this bug.

Description From 2005-04-28 16:54:06
Modules and SoftEnv are two tools used by users of Grids, like the TeraGrid, to
choose the software they need in their application environment.  Symbolic
software selection
is particularly important  for Grids because it standardizes software selection
and hides the
variability of where software is installed on individual grid resources. Users
can currently
describe software requirements before running applications in a shell and in
batch jobs,
but not thru GRAM execution or job services.

This is a request to extend the WS GRAM execution or job service RSL for GT 4.2
to include
a verb for symbolically describing what software is required by the spawned
process or job.

The TeraGrid is already working with the Condor project to make SoftEnv based
software selection available in Condor.
------- Comment #1 From 2005-05-18 16:01:46 -------
Can you give an example of what this might look like?  Is it truely just a
custom string that can be used in a hacked scheduler script?  If so, you could
probably do this now with a little work using the <extensions> element in GT4
GRAM's job description schema.
------- Comment #2 From 2005-05-18 23:44:36 -------
In RSL syntax the parameter would look something like:
  (software_environment=(@teragrid-basic +globus +mpich-g2))
Where the value is an order list of strings.
The Gatekeeper would need to execute something along the lines of
  source "`jobmanager-env-setup \"<above_value>\"`"
before executing the user specified program regardless of jobmanager
(fork, pbs, lsf, whatever).  The jobmanager-env-setup script would be
provided by GT but would need to be customized to implement environment
setup using SoftEnv of Modules.  Or perhaps better, we would need to
create two add-on packages, similary to the JobManager add-on packages fo
 PBS and other resource managers, to implement jobmanager-env-setup using
SoftEnv or Modules.  The critical element is that the setup must happen before
the user specified program is executed so that the environment variabled
defined when the program runs includes whatever is needed to use the
select software (@teragrid-basic +globus +mpich-g2 for example).
------- Comment #3 From 2005-10-19 12:02:37 -------
This is a critical feature for many large sites around the world, not "just"
------- Comment #4 From 2005-10-25 16:09:14 -------

What do you mean by your comment?  Can you describe what feature you are talking about?  Is it that 
you want integrated softenv like TG?

JP wants the softenv command(s) to be included in the scheduler command file after the scheduler 
commands but before the user application is executed.  Is that what you want?

Also, JP mentioned that softenv could be used to locate the scheduler commands called in the perl 
scripts to submit/cancel the job.  e.g. qsub, qdel, ...   This is problematic for TG as they install new PBS 
versions and the JM is still pointing to the old ones.  If the perl scripts used softenv, then the correct 
version would automatically be used.

------- Comment #5 From 2005-11-11 10:50:26 -------
Bumping the priority as TG would like this done by end of November to be 
included in the upcoming TG production release.
------- Comment #6 From 2005-11-11 11:39:15 -------
Just to be sure, this is NOT going to be added to the 4.0 branch. This is a
feature addition not a bug fix (I've changed the severity to "enhancement" as it
should be.).  If a CVS snapshot is to be used, then it concerns me that a TG
software release is relying on untested development software.
------- Comment #7 From 2005-11-11 11:43:19 -------
The TeraGrid isn't running stock GT 4.0.1.  We already have a collection of
Globus patches that
fix TeraGrid issues, and improvements that to address TeraGrid requirements,
all of which will
be tested on the TeraGrid before going production.
------- Comment #8 From 2005-11-11 16:49:07 -------
I was asked to add a comment, so I'll do that in all haste:

What JP describes makes perfect sense: as part of the Globus-generated 
submission script, do a callout to set the appropriate environment variables 
so that executables, libraries and licenses can be found. Note that this also 
requires a relaxation in that GRAM can not look for, and expect to find, the 
executable on beforehand.

I can assist in testing the integration to Modules, as that is what is used at 

Just to be clear: in addition to be something that is statically configured by 
the site admin only, the client should also be allowed to trigger this in the 
GRAM request, e.g. "&(software=gaussian-8.1)(executable=gaussian)"

I am already thinking of pushing this one step further: allowing for 
thÝs "software module" directive to not only set environment variables but 
also allow it to fill in / override any other element in the job description, 
such as the name of the executable to run. This would allow for a great deal 
of simplication on the client side: stage in the right set of input files, 
specify (software="myprog-1.0") and let the local sysadmin fill out the rest.

------- Comment #9 From 2005-11-14 12:52:48 -------
Can somebody provide documentation links for Modules and SoftEnv. I can't seem
to find anything with Google. Thanks.
------- Comment #10 From 2005-11-14 13:47:30 -------
Follow the SoftEnv links at http://www-unix.mcs.anl.gov/systems/software/msys/.
SoftEnv is an MCS systems developed tool. I'm currently supporting it for the TeraGrid.
If we talk directly about what and how to do it will save you some time. I'm available
any time starting next week.
------- Comment #11 From 2005-11-14 17:06:26 -------

Typical commands:
[~] $ module avail gcc

gcc/3.2.1(default) gcc/2.95.3 gcc/3.3.1

[~] $ module add gcc
[~] $ gcc --version
gcc (GCC) 3.2.1
Copyright (C) 2002 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO

[~] $ module add gcc/3.3.1
[~] $ gcc --version
gcc (GCC) 3.3.1
Copyright (C) 2003 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO

[~] $ 
------- Comment #12 From 2005-12-01 19:05:15 -------
I added a new Bugzilla entry (bug #3967) specifically for the TeraGrid SoftEnv
extensions. I'm not marking this as a dependency to this bug since it's tailored
for SoftEnv/is not generic and won't be distributed to anyone but TeraGrid.
------- Comment #13 From 2007-09-19 11:37:55 -------
Reassigning to current GRAM developer to close/fix as appropriate.
------- Comment #14 From 2008-02-04 14:21:37 -------
We've implemented a softenv specific solution for TeraGrid (and anyone using
softenv).  I don't see a more general solution happening in 4.2.  Moving to
4.4.  We'll reevaluate there.
------- Comment #15 From 2012-09-05 11:42:36 -------
Doing some bugzilla cleanup...  Resolving old GRAM3 and GRAM4 issues that are
no longer relevant since we've moved on to GRAM5.  Also, we're now tracking
issue in jira.  Any new issues should be added here: