Bug 5591 - Improve Java WS Core metrics reporting
: Improve Java WS Core metrics reporting
: Java WS Core
: unspecified
: All All
: P1 blocker
: 4.2.0
Assigned To:
  Show dependency treegraph
Reported: 2007-10-02 09:23 by
Modified: 2008-06-06 15:02 (History)



You need to log in before you can comment on or make changes to this bug.

Description From 2007-10-02 09:23:15
- Update reporting to be periodic rather than only at start-up and shut-down.
With trunk code base, the report could be triggered by deploy of service, since
remote deploy and shutdown of container is required at deploy. With 4.0.x it
will need be time-based periodic reporting.

- Update packet information to contain host/port, services list with
information to determine if service was just deployed or activated. Potentially
compression of service information might be requred. Evaluate other parameters
that would be useful for reporting.

- Phase out the current package type sent. The information sent can be absorbed
in the proposed new package and event type can be expanded to indicate periodic
update, in addition to the two sent in current deployment.
------- Comment #1 From 2007-10-02 11:00:57 -------
The MDS team would also like a ws core enhancement that does two things:

- tracks the number of operations (i.e. getRP, queryRP, etc) on a per-service
level, and
- provides a call (or simple api) that allows access to this information
------- Comment #2 From 2007-10-02 11:13:44 -------
I've bumped the priority of this enhancement request (don't know if that will
have any effect, but what the heck...) because this is critical in order for us
to learn how people are using MDS4 (index service usage behavior).  WIthout
this capability, the index service can not report on its own usage, other than
volume of data stored in the index.  

Note that this also just seems like a really good idea: it seems helpful for
any hosted service to be able to find out how much its resource properties are
being used.
------- Comment #3 From 2008-01-18 16:47:32 -------
Proposed changes and an updated packet has been proposed on the mailing lists
for comments.
http://www-unix.mcs.anl.gov/~ranantha/jwscoreMetricsUpdateJan18.txt has the

The proposed packate does not include any security information, like the
authorization system used or trusted certiticates. I suggest that such
information be in an independent packet, so deployers can explicitly turn it
off. This will ensure that deployers who don't want to report security
information are not dettered from reporting core usage packets.
------- Comment #4 From 2008-02-04 16:56:31 -------
This bug is now a subtask of a larger campaign to improve Java WS Core metrics
reporting in both 4.2.0 and 4.0.x releases.  This bug reflects the 4.2.0
portion of this effort.
------- Comment #5 From 2008-03-31 16:11:33 -------
This code for this deliverable has been complete at both the sending and
receiving sides since 3/6/08.  

In order to close out the issue for 4.2.0, the usage stats production database
needs to be updated by altering the java_ws_core_packets table so that it
contains the new fields.  I have not been able to complete this step myself as
it requires access to the database installation that I do not have.  

Any existing database administrator can perform this step by issuing the
following SQL statements against the usage stats database:

  ALTER TABLE java_ws_core_packets ADD version_major SMALLINT;
  ALTER TABLE java_ws_core_packets ADD version_minor SMALLINT;
  ALTER TABLE java_ws_core_packets ADD version_micro SMALLINT;
  ALTER TABLE java_ws_core_packets ADD port_number INT;
  ALTER TABLE java_ws_core_packets ADD thread_pool_size SMALLINT;
  ALTER TABLE java_ws_core_packets ADD thread_count SMALLINT;
  ALTER TABLE java_ws_core_packets ADD max_threads SMALLINT;
  ALTER TABLE java_ws_core_packets ADD threads_high_water_mark SMALLINT;
  ALTER TABLE java_ws_core_packets ADD service_request_count INT;
  ALTER TABLE java_ws_core_packets ADD jvm_info VARCHAR(64);

Once this step is performed, the sending of the new packet type can be enabled
in JWS-Core and the issue can then be closed.
------- Comment #6 From 2008-04-22 12:04:04 -------
Discussed with Mike and he has completed testing with a test database with the
new schema. Once the production database is altered, this work can be closed.
------- Comment #7 From 2008-04-28 22:41:34 -------
Packet sending code has been enabled in trunk.  This bug is no longer a blocker
for JWS-Core for 4.2 release.  The packet receiver handler and database schema
changes have both been tested successfully in a separate deployment (with the
current trunk code for metrics project) using Postgres as the database. 

To my knowledge, the production usage stats database and listener service at
usage-stats.globus.org have not yet been updated to reflect the new packet
version.  JWS-Core V3 packets will be reported by this listener as
'unrecognized' until an administrator modifies the production configuration.