Bug 6840 - The PBS job manager doesn't handle large environments well
: The PBS job manager doesn't handle large environments well
Status: RESOLVED WONTFIX
: GRAM
gt2 Gatekeeper/Jobmanager
: 4.0.8
: Open Science Grid (OSG) All
: P3 normal
: ---
Assigned To:
:
:
:
:
  Show dependency treegraph
 
Reported: 2009-08-19 13:28 by
Modified: 2012-09-12 10:48 (History)


Attachments


Note

You need to log in before you can comment on or make changes to this bug.


Description From 2009-08-19 13:28:09
Hi,

The VDT team recently got a bug report about the PBS jobmanager. I've heard of
the problem before, but not a bug fix for it. I'm not qualified to evaluate it
or suggest better code. Given that you maintain it, I hope you can take on this
bug. 

The problem is that submitting a job with a very large environment will fail. I
suspect the fix isn't quite right, though it works for this user's workloads. 

Here's the full text of his bug report:

> From: "Fengping Hu via RT" <vdt-support@opensciencegrid.org>
> Date: August 18, 2009 10:52:22 AM CDT
> Subject: [vdt-support #5688] A bug in pbs.pm?
> Reply-To: vdt-support@opensciencegrid.org
>
> Tue Aug 18 10:52:21 2009: Request 5688 was acted upon.
> Transaction: Ticket created by fhu@purdue.edu
>      Queue: vdt-support
>    Subject: A bug in pbs.pm?
>      Owner: Nobody
> Requestors: fhu@purdue.edu
>     Status: new
> Ticket URL: http://crt.cs.wisc.edu/Ticket/Display.html?id=5688
>
> ---------------------------------------------------------------------------
> Hi,
>
> Depend on what path the vdt location is, the pbs.pm could generate a pbs
> job script
> that is unsubmittable due to too long environment variable string to
> export.
>
> We've added three lines to the pbs.pm as a makeshift solution for this
> problem. But I think
> this probably should be addressed by officials.
>
> +    $env_count = @new_env;                                                                                      
> +    @new_env_part1 = splice(@new_env, 0, int($env_count/2) );                                                   
> +    print JOB '#PBS -v ' . join(',', @new_env_part1) . "\n";                                                    
>     print JOB '#PBS -v ' . join(',', @new_env) . "\n";                                                           
>
>
> Thanks,

You can see the full ticket at:

http://crt.cs.wisc.edu/Ticket/Display.html?user=guest&pass=guest&id=5688

What do you think?

Thanks,
-alain
-----------------------------------------------------------------
Alain Roy
Open Science Grid Software Coordinator            roy@cs.wisc.edu
http://opensciencegrid.org                 http://vdt.cs.wisc.edu
------- Comment #1 From 2009-08-19 15:14:49 -------
Hm, Fengping has retracted his suggestion: You can't have multiple PBS -v
lines. I guess we don't have a good suggestion at this point. :(
------- Comment #2 From 2009-08-19 15:25:55 -------
Is it the number of env vars or the overall length that is causing the problem?
 Why is in the env that is making it so large?  Maybe it is PBS configuration? 
Maybe a first step is to print out the env vars to a file in order to see what
all is in there.
------- Comment #3 From 2012-09-12 10:48:39 -------
We've migrated our issue tracking software to jira.globus.org. Any new issues
should be added here:

http://jira.globus.org/secure/VersionBoard.jspa?selectedProjectId=10363

As this issue hasn't been commented on in several years, we're closing it. If
you feel it is still relevant, please add it to jira.