Bug 5851 - Deletion of directories of persisted subscription resources
: Deletion of directories of persisted subscription resources
Status: RESOLVED FIXED
: Java WS Core
globus_wsrf_core
: alpha
: Macintosh All
: P2 major
: 4.2.0
Assigned To:
:
:
:
:
  Show dependency treegraph
 
Reported: 2008-02-06 11:38 by
Modified: 2008-04-04 07:28 (History)


Attachments


Note

You need to log in before you can comment on or make changes to this bug.


Description From 2008-02-06 11:38:33
This bug is somehow related to bug 5819.

Each persisted subscription resource seems to be stored in a separate
directory. When a subscription resource is destroyed via a WS call like

...
SubscriptionManager subscriptionPort =
  new WSBaseNotificationServiceAddressingLocator()
     .getSubscriptionManagerPort(subscriptionEndpoint);
...
subscriptionPort.unsubscribe(new Unsubscribe());

the persisted resource will be removed, but its directory keeps on
existing. I don't completely understand the use of the directory (it
did not exist in 4.0.x), so it might be that more than one resource is
to be stored in that directory and removal of that directory might be
critical?

Effect on persistence directories of containers with Gram4:
Gram4 subscribes for state changes of transfers in jobs with
staging/fileCleanUp and finally destroys the subscription resources.
This cannot be done via the SubscriptionRemovalHelper because RFT can
be located in a different container.
So there will be an orphaned directory per transfer/fileCleanUp request
after a job has been fully processed.

Regardless of Gram: This will effect persistence directories of all
containers where persistent subscriptions are in use by arbitrary
services that don't use the SubscriptionRemovalHelper.

There is a Todo-comment about that in PersistentSubscription.remove().
------- Comment #1 From 2008-03-21 15:07:18 -------
It looks like the addition of the directory-per-producer modification was
introduced in order to make the cleanup process easier for
SubscriptionRemovalHelper.java, since the code just needs to locate the
appropriate directory and then delete all of the files within.  

The original code had each resource (upon destruction) check the storage
directory, and delete it if empty.  This was subsequently removed as it could
cause potential race condition with another persistent subscription resource if
that resource also tried to remove or create the directory again at the same
time.

The existing code as commented suggests implementing a sweeper function to
periodically look for empty directories and remove them.  However, it seems
like this approach would also be prone to the same race condition.

The proposed solution is to remove the per-producer directory modification so
that the persistent directory structure is the same as 4.0.x series.  In other
words, there will be a single directory in which all of the persistent
subscriptions for the container will be stored.  

In order for this to work, the batch removal logic called by
SubscriptionRemovalHelper will have to modified to look for only the files
associated with the specified notification producer.  This should not be a
problem because in 4.2 the persisted object filename has the notification
producer key encoded into the filename, e.g.:

6D01551DC6F6D9E386FB98E76C112FA928627CBF+717f8df0-f788-11dc-ba3e-bcd76adc9d87.obj

In the above filename the string to the left of the "+" is the notification
producer key; so the strategy would be to obtain a list of the appropriate
files using Java FilenameFilter class, and then simply loop and delete them.
------- Comment #2 From 2008-03-25 16:34:53 -------
Fix committed to trunk.  Core notification tests all pass.  Martin, I will wait
for word from you on GRAM tests before closing this bug.
------- Comment #3 From 2008-03-26 14:14:59 -------
A simple job submission with staging showed that stuff is cleaned up
properly. didn't try a big submission yet.
------- Comment #4 From 2008-04-04 07:28:56 -------
Discussed with Martin, the fix worked, closing the bug.