Bug 5381 - Loss of precision for xsd:float and xsd:double
: Loss of precision for xsd:float and xsd:double
Status: RESOLVED FIXED
: C WS Core
C WSRF core
: 4.0.4
: PC Linux
: P3 normal
: 4.0.6
Assigned To:
:
:
:
:
  Show dependency treegraph
 
Reported: 2007-06-15 16:10 by
Modified: 2008-01-11 13:26 (History)


Attachments


Note

You need to log in before you can comment on or make changes to this bug.


Description From 2007-06-15 16:10:45
globus_soap_message_serialize.c converts a double to a string by using
sprintf("%E"). From the man page:

       e,E    The  double  argument  is  rounded  and  converted  in the style
              [-]d.dddeħdd where there is one digit before  the  decimal-point
              character and the number of digits after it is equal to the pre-
              cision; if the precision is missing, it is taken as  6

In other words, if the sender specifies an xsd:float or xsd:double as
0.97587222395
what goes on the wire is 
9.758722E-01

One possible remedy is to change the formatting string to something like "%15E"
or "%30E" or whatever people would consider an acceptable precision. 6 digits
is probably too little though.
------- Comment #1 From 2007-10-08 10:06:32 -------
Just to add some info, a 64 bit double needs 16 digits of precision for full
representation...
------- Comment #2 From 2007-11-30 13:31:32 -------
Committed fix to 4.0 and trunk