XRAY: How to resolve this vague escript error from rabbitmq/erlang

Nir Ovadia
2023-01-22 11:06

This article is mainly for if you see rabbitmq failing to start after an upgrade from Xray 2 -> 3. If you try to run the $XRAY_HOME/app/third-party/rabbitmq/sbin/rabbitmqctl
binary (for example running “rabbitmqctl version”) and it returns a escript: exception error: undefined function rabbitmqctl_escript:main/1
error then you’re hitting this issue. This is usually on more customized installs, so I recommend making sure that the rabbitmq dependencies (socat and erlang) have also been installed as part of the upgrade and aren’t missing.

This specific issue actually seems to be a bug in Rabbitmq or Erlang. The issue is due to a symlink within the path that the rabbitmq binary resides on, so for example if the default path is /opt/jfrog/xray/app/third-party/rabbitmq/sbin/rabbitmqctl
and there is a symlink from /opt/jfrog/xray -> /data/opt/jfrog/xray  so the actual path is /data/opt/jfrog/xray/app/third-party/rabbitmq/sbin/rabbitmqctl
then because the binary is on a symlink, it’ll fail. You can test it out by seeing that a /opt/jfrog/xray/app/third-party/rabbitmq/sbin/rabbitmqctl version fails but a /data/opt/jfrog/xray/app/third-party/rabbitmq/sbin/rabbitmqctl versionsucceeds, showing the symlink in the path is the issue. We sometimes see this in older custom setups, which often also have symlinks in the /xray/data directory. That is completely fine, the only issue is a symlink within the path to the rabbitmq binary. To resolve it, you would need to remove the symlink in this part of the filesystem. The binary location doesn’t have much data, it’s all in the $XRAY_HOME/var/data location, so symlinking the binaries isn’t saving data or anything.

There are a couple of reports on this https://github.com/rabbitmq/rabbitmq-server/issues/1679

but the behavior itself isn’t always very clear, so if you upgrade and are having issues it’s a good thing to run a quick check on the rabbitmq binary or look into possible symlinks to make sure that isn’t an issue.