Bugzilla – Bug 5107
LIGO: globus-rls-server does not have automatic reconnection to DBMS
Last modified: 2008-05-28 19:19:15
You need to
before you can comment on or make changes to this bug.
In newer versions of MySQL (post version 3, I believe), the globus-rls-server
disconnects from the MySQL database after a short amount of time. This time can
be changed with configuration options to the MySQL server itself, but it cannot
make RLS automatically reconnect. Therefore, after whatever amount of time
specified of no MySQL activity, the RLS server needs to be restarted to
reattach itself to MySQL.
An example error message (4.1 of RLS with 5.0.22 of MySQL and 2.2.11 of
globus_rls_client: DB error: [MySQL][ODBC 3.51
Driver][mysqld-5.0.22-standard]MySQL server has gone away
The MySQL documentation says this about the error:
'You have encountered a timeout on the server side and the automatic
reconnection in the client is disabled (the reconnect flag in the MYSQL
structure is equal to 0).'
It doesn't make sense to make large timeouts globally for all MySQL clients.
Instead, a service like RLS which is bound to the database backend should
always be able to reattach itself to the database if it has lost its
As a side note, I corresponded with some RLS developers in August 2006 about
this but never saw anything further about this issue.
I have recreated the bug with:
MySQL Connector/ODBC 3.51.15
One interesting finding is that the old ODBC driver, MyODBC 3.51.06, seems to
have an auto-reconnect feature enabled by default. When using that driver, the
odbc driver reconnects and the RLS does not experience this bug.
MySQL has an auto-reconnect option available for MySQL Connector/ODBC driver's
starting with version 3.51.13.
option = 4194304
"Enables auto-reconnection functionality. You should not use this option with
transactions, since a auto reconnection during a incomplete transaction may
cause corruption. Note that an auto-reconnected connection will not inherit the
same settings and environment as the original. This option was enabled in
I have tested it with 3.51.15 with an odbc.ini that looks like this:
Database = lrc1000
SERVER = myhost.isi.edu
PORT = 3306
option = 4194304
Using this option the RLS server (rather, the odbc driver) reconnects to the
Comment #c2 discusses a partial solution based on the MySQL ODBC driver
auto-reconnect feature. Alternatively, it may be desirable to implement a
solution within the RLS server that uses the ODBC interface alone (as opposed
to relying on the configuration of the ODBC driver).
Using only the ODBC interface, one convenient solution would be to check the
connection attribute SQL_ATTR_CONNECTION_DEAD using the method
SQLGetConnectAttr. If a check on the connection dead attribute returned true,
then RLS would close/free the connection resources and reconnect.
Unfortunately, the attribute (SQL_ATTR_CONNECTION_DEAD) describe above is
available in the ODBC 3.5 standard specification, which is not supported by the
open source (permissively licensed) ODBC interface that RLS depends on --
libiodbc. Currently, libiodbc is ODBC 3.0 compliant.
A less ideal alternative may be to attempt a simple, dummy query to test
whether the connection is alive.
In the interest of finding a solution for PostgreSQL (as well as MySQL) users,
I have recreated the bug with:
Just noticed that there was a bug in MySQL drivers from >=3.51.10 to <3.51.20
that prevents dead connections from being properly identified. Full details are
So for the SQL_ATTR_CONNECTION_DEAD attribute to respond correctly MySQL users
will need to use 3.51.20+ drivers.
The following patch (actually one for 4.0 branch and one for trunk) enable RLS
server to reconnect to a db.
4.0 branch: http://www.isi.edu/~schuler/bug5107/bug5107.40branch.diff
Both are committed to CVS.
The RLS now uses the SQLGetConnectionAttr call to check the
SQL_ATTR_CONNECTION_DEAD attribute. It appears that we can expect recent
versions of unixODBC, iodbc, MySQL and SQLite drivers to support it.
On each new request, the RLS checks the connection dead attribute. If the
driver indicates that the connection is dead, the RLS closes the connection,
frees it, and then attempts to open a new database connection. If it fails to
open, that is an indication that the db is down, and the RLS returns a
GLOBUS_RLS_DBERROR to the client.
It has been tested with:
MySQL ODBC 3.51.06
MySQL ODBC 3.51.15
MySQL ODBC 3.51.21
SQLite ODBC 0.74
On MySQL drivers 3.51.20+ support the connection attribute properly. For
-- 3.51.06 does not support it but automatically reconnects by default
-- 3.51.15 does not support it and does not automatically reconnect by
-- 3.51.21 does support it and does not automatically reconnect by default
As noted earlier, MySQL has an option that tells the drivers to (such as
3.51.15) to automatically reconnect. MySQL users should upgrade their drivers
to a supporting version or set the auto-reconnect option.
A GPT package of the updated globus_rls_server:
I believe this will be added to the GT Advisories page soon.
The package is now available on the GT Advisories page.