Bugzilla – Bug 4699
RFT performance improvements
Last modified: 2009-03-20 13:50:26
You need to
before you can comment on or make changes to this bug.
Title: RFT performance improvements
During performance tests of large job submissions with GRAM4 and
Condor-G we found that RFT does not remove the restart marker records
from the table 'restart' or the RFT database.
This does not cause problems in a fresh database setup of RFT but
increasingly causes staging processes to take much longer than required
if this table gets filled up since a query on this table is done during
each staging process.
Ravi fixed this already in 4.0.branch code. But 4.0.3 does not include
that fix. So this is a more formal campaign to have a good location where
a patch for 4.0.3 code can be deposited.
Created an attachment (id=1044) [details]
new logging class
Created an attachment (id=1045) [details]
In the last two comments I attached are two files: a new java code file
'MyPerformanceLog.java' (new logging class) and a patch for GT 4.0.3
to fix the problem.
First copy 'MyPerformanceLog.java' to
Then the patch 'javaCode.patch' must be applied to the code in the directory
patch < javaCode.patch
If not already done all records of the table 'restart' in the rft database
should be removed. SQL command for this after a successful login to the
database system and selection of the database 'rft_database':
delete from restart;
I understand this has been committed to the 4 0 branch, so will be in 4.0.4.
But still needs to be committed to the trunk.
Martin, Ravi, what more needs to be done here?
I don't remember this at all, was it really me who wrote this? (-:
I guess this was the problem with the 800000 records in the RFT
I didn't commit anything to RFT regarding this topic.
But there are still some database issues. I can't precisely say
what goes wrong, but the database still heaps up, not as dramatically
as last time, but a little bit. I assume that records don't get cleaned
up when transfers are interrupted. So maybe something with the lifetime
of transfer resource goes wrong???
This fix has been released in 4.0.4, 4.0.5 and 4.0.6. Marking Fixed.