Bugzilla – Bug 5840
Adding priorities to internal job states regarding processing order
Last modified: 2008-02-06 14:19:31
You need to log in before you can comment on or make changes to this bug.
Currently the queue that stores job resource keys of resources that need further processing is a FIFO queue. In case 1000 new jobs are queued this means that they are all processed in state None, then all jobs are processed in state Start, etc. and that the first job finishes quite late. This is not very effective regarding throughput. I changed this queue to be a priority queue so that jobs with the highest priority are processed first. The priority of a job resource depends on its internal state: it goes up the more advanced it is in its internal state. Advantages: * Tests with short-running jobs submitted by Condor-G showed that the throughput is better and that the container does not heap up so many resources this way because jobs finish right from the beginning and can be destroyed by a client. * This might also be better in a termination of a large job submission. If many jobs are still in state None while still some are being processed, those jobs in state None don't need cleanup but can be wiped out immediately. * There's also a psychological effect here: With the old model the user might have received the impression that damn Gram4 does not get a single job done until the very end in a big submission :-)