Bugzilla – Full Text Bug Listing
|Summary:||RFT performance improvements|
|Product:||RFT||Reporter:||Martin Feller <email@example.com>|
|Component:||Campaign||Assignee:||Ravi Madduri <firstname.lastname@example.org>|
|Severity:||normal||CC:||email@example.com, firstname.lastname@example.org, email@example.com, firstname.lastname@example.org, email@example.com, firstname.lastname@example.org|
|Bug Depends on:|
new logging class
Title: RFT performance improvements During performance tests of large job submissions with GRAM4 and Condor-G we found that RFT does not remove the restart marker records from the table 'restart' or the RFT database. This does not cause problems in a fresh database setup of RFT but increasingly causes staging processes to take much longer than required if this table gets filled up since a query on this table is done during each staging process. Ravi fixed this already in 4.0.branch code. But 4.0.3 does not include that fix. So this is a more formal campaign to have a good location where a patch for 4.0.3 code can be deposited.
Created an attachment (id=1044) [details] new logging class
In the last two comments I attached are two files: a new java code file 'MyPerformanceLog.java' (new logging class) and a patch for GT 4.0.3 to fix the problem. First copy 'MyPerformanceLog.java' to ws-transfer/reliable/service/java/source/src/org/globus/transfer/reliable/service/ Then the patch 'javaCode.patch' must be applied to the code in the directory ws-transfer/reliable/service/java/source/ with patch < javaCode.patch If not already done all records of the table 'restart' in the rft database should be removed. SQL command for this after a successful login to the database system and selection of the database 'rft_database': delete from restart;
I understand this has been committed to the 4 0 branch, so will be in 4.0.4. But still needs to be committed to the trunk. -Stu
Martin, Ravi, what more needs to be done here?
I don't remember this at all, was it really me who wrote this? (-: I guess this was the problem with the 800000 records in the RFT database, right? I didn't commit anything to RFT regarding this topic. But there are still some database issues. I can't precisely say what goes wrong, but the database still heaps up, not as dramatically as last time, but a little bit. I assume that records don't get cleaned up when transfers are interrupted. So maybe something with the lifetime of transfer resource goes wrong???