Bugzilla – Bug 5838
Share a threadpool for SEG event processing
Last modified: 2008-02-06 14:18:22
You need to log in before you can comment on or make changes to this bug.
Currently every JobStateMonitor has a queue and a thread that dispatches events provided by the SEG to the ManagedExecutableJobHome (implements JobStateChangeListener). It seems more efficient to have a common threadpool that handles events from all SEG's (Fork, PBS, Condor, ...). I added a JNDI configuration option which let a admin configure the size of this pool (default is 2 threads). By this we make sure that we don't waste a thread idling around if e.g. no Fork jobs come in and thus no Fork events have to be dispatched while the PBS event dispatcher thread one is really busy because a lot of PBS jobs have been submitted. Two situations make it desirable to increase the number of SEG event dispatching threads: 1. We can't anticipate how a self-written SEG will work. It might produce periodical events and much more than one thread can handle properly. 2. If the persistence directory is located no a slow NFS partition the dispatching of events may take long due to us persisting the local job state and if many events come in the queue heaps up and job processing will be delayed in the container. It feels like the jobs keep stuck in such a situation but it's just that we don't dispatch events quickly enough. Both situations caused problems on a thread in gt-user