ARTIFACTORY: How to resolve the oracle DB driver error with metadata service after upgrading to version 7.55.x and above in kubernetes with splitServicesToContainers

The splitServicesToContainers options will allow artifactory microservice to run on individual containers. It was enabled by default since version 107.60.0, but can also be turned on before that. If you have enabled the option since artifactory version 7.55.x and are using Oracle database, you may run into the following oracle DB driver issue with the metadata service.

[jfmd ] [WARN ] [ ] [jobs.go:98 ] [main ] - [RETRY] Initiating database connection: ORA-00000: DPI-1047: Cannot locate a 64-bit Oracle Client library: "libclntsh.so: cannot open shared object file: No such file or directory". See https://oracle.github.io/odpi/doc/installation.html#linux for help. Retrying in 1s ... (13/120) [database]

The root cause could be related to how you install the oracle DB driver with preStartCommand and that some lib.so file has changed location since version 7.55.x

Identify the issue

If you do see the above error with splitServicesToContainers option enabled, check if your artifactory.preStartCommand in values.yaml looks similar to this:

preStartCommand: "mkdir -p /opt/jfrog/artifactory/var/bootstrap/artifactory/tomcat/lib; cd /opt/jfrog/artifactory/var/bootstrap/artifactory/tomcat/lib && curl -o instantclient-basic-linux.x64-21.8.0.0.0dbru.zip https://download.oracle.com/otn_software/linux/instantclient/218000/instantclient-basic-linux.x64-21.8.0.0.0dbru.zip && unzip -jn instantclient-basic-linux.x64-21.8.0.0.0dbru.zip && cp /opt/jfrog/artifactory/app/artifactory/libaio/* . && ln -s /usr/lib64/libnsl.so.2 /opt/jfrog/artifactory/var/bootstrap/artifactory/tomcat/lib/libnsl.so.1 && ls -lah && rm /opt/jfrog/artifactory/var/bootstrap/artifactory/tomcat/lib/instantclient-basic-linux.x64-21.8.0.0.0dbru.zip && cp * /opt/jfrog/artifactory/app/artifactory/libaio/"

Note that /usr/lib64/libnsl.so.2 has changed location from /usr/lib64 to /opt/jfrog/artifactory/app/artifactory/libaio since artifactory version 7.55.x.The above command will fail to link the lib file. It won’t cause an issue when all the microservices are in the same container as the prestart command would still cp the entire libaio folder, but if you enabled splitServicesToContainers, you might see complaints about missing libraries.

Solution

The following setting should help you get rid of the errors.

artifactory:
  preStartCommand:  mkdir -p /opt/jfrog/artifactory/var/bootstrap/artifactory/tomcat/lib; cd /opt/jfrog/artifactory/var/bootstrap/artifactory/tomcat/lib && curl https://download.oracle.com/otn_software/linux/instantclient/2111000/instantclient-basic-linux.x64-21.11.0.0.0dbru.zip -o instantclient-basic-linux.x64-21.11.0.0.0dbru.zip && unzip -jn instantclient-basic-linux.x64-21.11.0.0.0dbru.zip && cp instantclient_21_11/ojdbc8.jar . && cp /opt/jfrog/artifactory/app/artifactory/libaio/* .
  extraEnvironmentVariables:
  - name: LD_LIBRARY_PATH
     value: /opt/jfrog/artifactory/var/bootstrap/artifactory/tomcat/lib
metadata:
  extraEnvironmentVariables:
  - name: LD_LIBRARY_PATH
     value: /opt/jfrog/artifactory/app/artifactory/libaio/:/opt/jfrog/artifactory/var/bootstrap/artifactory/tomcat/lib