Bugzilla – Bug 5851
Deletion of directories of persisted subscription resources
Last modified: 2008-04-04 07:28:56
You need to log in before you can comment on or make changes to this bug.
This bug is somehow related to bug 5819. Each persisted subscription resource seems to be stored in a separate directory. When a subscription resource is destroyed via a WS call like ... SubscriptionManager subscriptionPort = new WSBaseNotificationServiceAddressingLocator() .getSubscriptionManagerPort(subscriptionEndpoint); ... subscriptionPort.unsubscribe(new Unsubscribe()); the persisted resource will be removed, but its directory keeps on existing. I don't completely understand the use of the directory (it did not exist in 4.0.x), so it might be that more than one resource is to be stored in that directory and removal of that directory might be critical? Effect on persistence directories of containers with Gram4: Gram4 subscribes for state changes of transfers in jobs with staging/fileCleanUp and finally destroys the subscription resources. This cannot be done via the SubscriptionRemovalHelper because RFT can be located in a different container. So there will be an orphaned directory per transfer/fileCleanUp request after a job has been fully processed. Regardless of Gram: This will effect persistence directories of all containers where persistent subscriptions are in use by arbitrary services that don't use the SubscriptionRemovalHelper. There is a Todo-comment about that in PersistentSubscription.remove().
It looks like the addition of the directory-per-producer modification was introduced in order to make the cleanup process easier for SubscriptionRemovalHelper.java, since the code just needs to locate the appropriate directory and then delete all of the files within. The original code had each resource (upon destruction) check the storage directory, and delete it if empty. This was subsequently removed as it could cause potential race condition with another persistent subscription resource if that resource also tried to remove or create the directory again at the same time. The existing code as commented suggests implementing a sweeper function to periodically look for empty directories and remove them. However, it seems like this approach would also be prone to the same race condition. The proposed solution is to remove the per-producer directory modification so that the persistent directory structure is the same as 4.0.x series. In other words, there will be a single directory in which all of the persistent subscriptions for the container will be stored. In order for this to work, the batch removal logic called by SubscriptionRemovalHelper will have to modified to look for only the files associated with the specified notification producer. This should not be a problem because in 4.2 the persisted object filename has the notification producer key encoded into the filename, e.g.: 6D01551DC6F6D9E386FB98E76C112FA928627CBF+717f8df0-f788-11dc-ba3e-bcd76adc9d87.obj In the above filename the string to the left of the "+" is the notification producer key; so the strategy would be to obtain a list of the appropriate files using Java FilenameFilter class, and then simply loop and delete them.
Fix committed to trunk. Core notification tests all pass. Martin, I will wait for word from you on GRAM tests before closing this bug.
A simple job submission with staging showed that stuff is cleaned up properly. didn't try a big submission yet.
Discussed with Martin, the fix worked, closing the bug.