2764 2005-02-18 10:39 Remove Postgresql specific queries from RFT 2005-06-14 11:21:16 1 1 1 Unclassified RFT RFT development Macintosh All RESOLVED FIXED P3 enhancement 4.2 1 madduri@mcs.anl.gov madduri@mcs.anl.gov ghulamhu@us.ibm.com millerjj@us.ibm.com nitswamy@in.ibm.com ogsa-bugs@globus.org paxhia@us.ibm.com seelbach@us.ibm.com tboehm@de.ibm.com madduri@mcs.anl.gov 2005-02-18 10:39:04 Find a way to generate primary keys instead of using database to do that. For eg: i can have a UUID as request_id and use it in my inserts. paxhia@us.ibm.com 2005-03-01 11:20:55 I agree GT4 should be database neutral. I don't think you need to find another way to generate IDs, or use other identifiers like UUID as an index into the requests. Postgres, MySql, Cloudscape/Derby and other databases have ways of generating sequential IDs. This is usually defined when you create the tables. In GT4, in the ReliableFileTransferDbAdapter.java code you reference a Postgres extension (ie. SEQUENCE) with the following statement: ResultSet rs = statement.executeQuery("SELECT last_value FROM " + "request_seq"); In GT3, in the TransferDbAdapter.java code you use a more database generic statement: ResultSet rs = statement.executeQuery("SELECT COUNT(id) FROM " + "request"); Why not use this more generic mechanism? madduri@mcs.anl.gov 2005-03-01 12:56:38 Because when rft resources reach end of their life time they are removed from the database and select count(id) from request; may result in a id that is already been used. it worked in gt3.2 as the records in the database were never cleaned by the service bacon@mcs.anl.gov 2005-03-04 14:01:53 Via the email gateway: Ravi I have a few doubts about your response and would like further clarification regarding this. Lets consider the following steps: (1) state of a transfer request is active now. (2) the database is accessed and a row is created (3) Lets assume the autoincrement column is used to generate ids so an id is chosen (4) the service has the id of the request and it can use that id to refer to the respective row in the database (5) The service uses the id by doing select count(id), lets say in the following situations: -> when the marker is changed -> when the container goes down and comes back up again -> when the lifetime of the request expires (6) Finally, in an ideal case the transfer request reaches the "Finished" state and a transfer is successfully completed. (7) The service starts the cleanup process.. Now I have no idea how does this cleanup process works,, like what are the details. How does this cleanup process makes the use of sequence in Postgres safer, agreeable and feasible and why and how does it make the count(id) problematic? Probably I am not understanding exactly what are the pros and cons of one technique over another? We agree that the service persists its state data, hence the ids its keeping track of, is aware of, is using to access the db. What I dont understand is how the service persists the id data, if sequence is used and why it wont persist the id data if count(id) is used and what are the impacts of the cleanup process on either of these techniques? I really appreciate some further clarification regarding this issue. Thanks a lot, Shama Ghulamhussain IBM Advanced Systems Infrastructure Development T/L: 295 8284 ghulamhu@us.ibm.com madduri@mcs.anl.gov 2005-03-18 18:47:28 Created an attachment (id=539) removing select last_value from request_seq query Ok this is what you got to do. 1.remove request_seq from rft database schema. 2. Apply this patch if possible or do something that I do here. You basically should insert the requestId instead of database inserting it with next value of sequence. 3. See if you can get away doing the same trick with transfer table. And let me know how it works out. madduri@mcs.anl.gov 2005-04-01 16:46:51 Any updates on this one ? madduri@mcs.anl.gov 2005-06-14 11:21:16 I just comitted changes to trunk and globus_4_0_branch that would work around from using sequences in postgres. I tested my changes with mysql. 539 2005-03-18 18:47 removing select last_value from request_seq query diff text/plain 2296 madduri@mcs.anl.gov SW5kZXg6IHNyYy9vcmcvZ2xvYnVzL3RyYW5zZmVyL3JlbGlhYmxlL3NlcnZpY2UvZGF0YWJhc2Uv UmVsaWFibGVGaWxlVHJhbnNmZXJEYkFkYXB0ZXIuamF2YQo9PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09ClJDUyBmaWxlOiAv aG9tZS9nbG9iZGV2L0NWUy9nbG9idXMtcGFja2FnZXMvd3MtdHJhbnNmZXIvcmVsaWFibGUvc2Vy dmljZS9qYXZhL3NvdXJjZS9zcmMvb3JnL2dsb2J1cy90cmFuc2Zlci9yZWxpYWJsZS9zZXJ2aWNl L2RhdGFiYXNlL1JlbGlhYmxlRmlsZVRyYW5zZmVyRGJBZGFwdGVyLmphdmEsdgpyZXRyaWV2aW5n IHJldmlzaW9uIDEuNDQKZGlmZiAtdSAtcjEuNDQgUmVsaWFibGVGaWxlVHJhbnNmZXJEYkFkYXB0 ZXIuamF2YQotLS0gc3JjL29yZy9nbG9idXMvdHJhbnNmZXIvcmVsaWFibGUvc2VydmljZS9kYXRh YmFzZS9SZWxpYWJsZUZpbGVUcmFuc2ZlckRiQWRhcHRlci5qYXZhCTE1IE1hciAyMDA1IDIzOjA2 OjQyIC0wMDAwCTEuNDQKKysrIHNyYy9vcmcvZ2xvYnVzL3RyYW5zZmVyL3JlbGlhYmxlL3NlcnZp Y2UvZGF0YWJhc2UvUmVsaWFibGVGaWxlVHJhbnNmZXJEYkFkYXB0ZXIuamF2YQkxOSBNYXIgMjAw NSAwMDo0MTozNCAtMDAwMApAQCAtMTI5LDIzICsxMjksMjUgQEAKICAgICAgICAgICAgICAgICAg ICAgcmVxdWVzdC5nZXRUcmFuc2ZlckNyZWRlbnRpYWxFbmRwb2ludCgpLCBxTmFtZSk7CiAgICAg ICAgICAgICBjID0gUkZURGF0YWJhc2VTZXR1cC5nZXREQkNvbm5lY3Rpb24oKTsKICAgICAgICAg ICAgIHN0YXRlbWVudCA9IGMuY3JlYXRlU3RhdGVtZW50KCk7CisgICAgICAgICAgICBSZXN1bHRT ZXQgcnMgPSAKKyAgICAgICAgICAgICAgICBzdGF0ZW1lbnQuZXhlY3V0ZVF1ZXJ5KCJTRUxFQ1Qg Y291bnQoaWQpIGZyb20gcmVxdWVzdCIpOworICAgICAgICAgICAgd2hpbGUgKHJzICE9IG51bGwg JiYgcnMubmV4dCgpKSB7CisgICAgICAgICAgICAgICAgcmVxdWVzdElkID0gcnMuZ2V0SW50KDEp OworICAgICAgICAgICAgfQorCiAgICAgICAgICAgICB0aGlzLnF1ZXJ5Q29uc3RhbnQgPSBzdGF0 ZW1lbnQKLSAgICAgICAgICAgICAgICAgICAgLmV4ZWN1dGVVcGRhdGUoIklOU0VSVCBJTlRPIHJl cXVlc3QoY29uY3VycmVuY3ksIgorICAgICAgICAgICAgICAgICAgICAuZXhlY3V0ZVVwZGF0ZSgi SU5TRVJUIElOVE8gcmVxdWVzdChpZCwgY29uY3VycmVuY3ksIgogICAgICAgICAgICAgICAgICAg ICAgICAgICAgICsgInRlcm1pbmF0aW9uX3RpbWUsIiArICJhbGxfb3Jfbm9uZSxtYXhBdHRlbXB0 cywiCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgKyAidXNlcm5hbWUsc3RhcnRlZCxkZWxl Z2F0ZWRfZXByLHVzZXJfc3ViamVjdCkiIAotICAgICAgICAgICAgICAgICAgICAgICAgICAgICsg IiBWQUxVRVMgKCIgKyBjb25jdXJyZW5jeSArICIsIgorICAgICAgICAgICAgICAgICAgICAgICAg ICAgICsgIiBWQUxVRVMgKCIgKyByZXF1ZXN0SWQgKyAiLCIgIAorICAgICAgICAgICAgICAgICAg ICAgICAgICAgICsgY29uY3VycmVuY3kgKyAiLCIKICAgICAgICAgICAgICAgICAgICAgICAgICAg ICArIHRlcm1pbmF0aW9uVGltZS5nZXRUaW1lSW5NaWxsaXMoKSArICIsIgogICAgICAgICAgICAg ICAgICAgICAgICAgICAgICsgYWxsT3JOb25lLmJvb2xlYW5WYWx1ZSgpICsgIiwiCiAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgKyByZXF1ZXN0LmdldE1heEF0dGVtcHRzKCkgKyAiLCciICsg dXNlcm5hbWUgKyAiJywiCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgKyAiZmFsc2UsJyIr IGRlbGVnYXRpb25FUFJTdHJpbmcKICAgICAgICAgICAgICAgICAgICAgICAgICAgICArICInLCci ICsgdXNlclN1YmplY3QgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgKyAiJykiKTsKLSAg ICAgICAgICAgIFJlc3VsdFNldCBycyA9IHN0YXRlbWVudC5leGVjdXRlUXVlcnkoIlNFTEVDVCBs YXN0X3ZhbHVlICBGUk9NICIKLSAgICAgICAgICAgICAgICAgICAgKyAicmVxdWVzdF9zZXEiKTsK IAotICAgICAgICAgICAgd2hpbGUgKHJzICE9IG51bGwgJiYgcnMubmV4dCgpKSB7Ci0gICAgICAg ICAgICAgICAgcmVxdWVzdElkID0gcnMuZ2V0SW50KDEpOwotICAgICAgICAgICAgfQogICAgICAg ICAgICAgdGhpcy5xdWVyeUNvbnN0YW50ID0gc3RvcmVUcmFuc2ZlcnMocmVxdWVzdElkLCByZXF1 ZXN0KTsKICAgICAgICAgICAgIHJldHVybiByZXF1ZXN0SWQ7CiAgICAgICAgIH0gY2F0Y2ggKFNR TEV4Y2VwdGlvbiBlKSB7Cg==