Bug 4699

Summary:	RFT performance improvements
Product:	RFT	Reporter:	Martin Feller <feller@mcs.anl.gov>
Component:	Campaign	Assignee:	Ravi Madduri <madduri@mcs.anl.gov>
Status:	RESOLVED FIXED
Severity:	normal	CC:	ajay_net1@hotmail.com, allcock@mcs.anl.gov, feller@mcs.anl.gov, gawor@mcs.anl.gov, lane@mcs.anl.gov, smartin@mcs.anl.gov
Priority:	P3
Version:	development
Target Milestone:	4.0.4
Hardware:	PC
OS:	Linux
Bug Depends on:
Bug Blocks:	4664
Attachments:	new logging class patch

Description From Martin Feller 2006-09-14 01:39:21

Title: RFT performance improvements

During performance tests of large job submissions with GRAM4 and
Condor-G we found that RFT does not remove the restart marker records
from the table 'restart' or the RFT database. 
This does not cause problems in a fresh database setup of RFT but
increasingly causes staging processes to take much longer than required
if this table gets filled up since a query on this table is done during
each staging process.

Ravi fixed this already in 4.0.branch code. But 4.0.3 does not include
that fix. So this is a more formal campaign to have a good location where
a patch for 4.0.3 code can be deposited.

------- Comment #1 From Martin Feller 2006-09-14 01:43:41 -------

Created an attachment (id=1044) [details]
new logging class

------- Comment #2 From Martin Feller 2006-09-14 01:44:50 -------

Created an attachment (id=1045) [details]
patch

------- Comment #3 From Martin Feller 2006-09-14 01:47:10 -------

In the last two comments I attached are two files: a new java code file
'MyPerformanceLog.java' (new logging class) and a patch for GT 4.0.3
to fix the problem.

First copy 'MyPerformanceLog.java' to
ws-transfer/reliable/service/java/source/src/org/globus/transfer/reliable/service/

Then the patch 'javaCode.patch' must be applied to the code in the directory
ws-transfer/reliable/service/java/source/ with

       patch < javaCode.patch


If not already done all records of the table 'restart' in the rft database
should be removed. SQL command for this after a successful login to the
database system and selection of the database 'rft_database':

           delete from restart;

------- Comment #4 From Stuart Martin 2006-10-10 12:41:43 -------

I understand this has been committed to the 4 0 branch, so will be in 4.0.4. 
But still needs to be committed to the trunk.

-Stu

------- Comment #5 From Stuart Martin 2007-08-18 14:14:48 -------

Martin, Ravi, what more needs to be done here?

------- Comment #6 From Martin Feller 2007-08-18 15:30:24 -------

I don't remember this at all, was it really me who wrote this? (-:
I guess this was the problem with the 800000 records in the RFT
database, right?
I didn't commit anything to RFT regarding this topic.
But there are still some database issues. I can't precisely say
what goes wrong, but the database still heaps up, not as dramatically
as last time, but a little bit. I assume that records don't get cleaned
up when transfers are interrupted. So maybe something with the lifetime
of transfer resource goes wrong???

------- Comment #7 From Stuart Martin 2008-02-04 11:09:36 -------

This fix has been released in 4.0.4, 4.0.5 and 4.0.6.  Marking Fixed.