Bugzilla – Bug 4699
RFT performance improvements
Last modified: 2009-03-20 13:50:26
You need to log in before you can comment on or make changes to this bug.
Title: RFT performance improvements During performance tests of large job submissions with GRAM4 and Condor-G we found that RFT does not remove the restart marker records from the table 'restart' or the RFT database. This does not cause problems in a fresh database setup of RFT but increasingly causes staging processes to take much longer than required if this table gets filled up since a query on this table is done during each staging process. Ravi fixed this already in 4.0.branch code. But 4.0.3 does not include that fix. So this is a more formal campaign to have a good location where a patch for 4.0.3 code can be deposited.
Created an attachment (id=1044) [details] new logging class
Created an attachment (id=1045) [details] patch
In the last two comments I attached are two files: a new java code file 'MyPerformanceLog.java' (new logging class) and a patch for GT 4.0.3 to fix the problem. First copy 'MyPerformanceLog.java' to ws-transfer/reliable/service/java/source/src/org/globus/transfer/reliable/service/ Then the patch 'javaCode.patch' must be applied to the code in the directory ws-transfer/reliable/service/java/source/ with patch < javaCode.patch If not already done all records of the table 'restart' in the rft database should be removed. SQL command for this after a successful login to the database system and selection of the database 'rft_database': delete from restart;
I understand this has been committed to the 4 0 branch, so will be in 4.0.4. But still needs to be committed to the trunk. -Stu
Martin, Ravi, what more needs to be done here?
I don't remember this at all, was it really me who wrote this? (-: I guess this was the problem with the 800000 records in the RFT database, right? I didn't commit anything to RFT regarding this topic. But there are still some database issues. I can't precisely say what goes wrong, but the database still heaps up, not as dramatically as last time, but a little bit. I assume that records don't get cleaned up when transfers are interrupted. So maybe something with the lifetime of transfer resource goes wrong???
This fix has been released in 4.0.4, 4.0.5 and 4.0.6. Marking Fixed.
