ARTIFACTORY: How to troubleshoot some common Artifactory startup failure issues

Background

After we install Artifactory or modify the configuration file, we may meet some startup failure issues due to some system problems or incorrect configuration file modifications. The following are some common cases.

Example 1

Access to internal network ports is restricted between JFrog Platform microservices:

Artifactory will use some Network Ports when starting up. If the firewall is turned on in the system, the health check between Artifactory microservices may fail during Artifactory startup. console.log reports a lot of messages such as:

2023-10-07T03:15:59.033Z [jfevt] [INFO ] [64c500894535962b] [access_thin_client.go:101     ] [main                ] [                ] - Cluster join: Retry 10: Service registry ping failed, will retry. Error: Error while trying to connect to local router at address 'http://localhost:8046/access/api/v1/system/ping': Get "http://localhost:8046/access/api/v1/system/ping": dial tcp [::1]:8046: connect: connection refused [startup]
…

Solution

Use the command "iptables -L" to check firewall rules and allow specific ports or stop the firewall. For example(Centos 7):

$ systemctl stop filewalld

Example 2

Format issue for the configuration file “system.yaml”:

If some parameters are not added according to the yaml format, such as:

configVersion: 1
shared:
node:
database:
  xxx
  …

Using this configuration, Artifactory will be unable to connect to the database, causing the access service to fail to start, and the console.log will report the following messages:

2023-10-07T03:37:35.760Z [jfrou] [INFO ] [078554a1333f4c8b] [join_executor.go:175          ] [main                ] [] - Cluster join: Retry 75: Service registry ping failed, will retry. Error: Get "http://localhost:8040/access/api/v1/system/ping": dial tcp [::1]:8040: connect: connection refused

Solution

Modify the configuration file using the correct format and restart Artifactory.

configVersion: 1
shared:
node:
database:
      xxx
        …

Example 3

Format issue for the configuration file “logback.xml”:

If the debug log configuration is added without following the XML format, such as:

<level value="debug">

The console.log reports errors such as:

11:06:04,109 |-ERROR in ch.qos.logback.core.joran.event.SaxEventRecorder@645209c3 - XML_PARSING - Parsing fatal error on line 368 and column 7
11:06:04,109 |-ERROR in ch.qos.logback.core.joran.event.SaxEventRecorder@645209c3 - org.xml.sax.SAXParseException; systemId: file:/opt/jfrog/artifactory/var/etc/artifactory/logback.xml; lineNumber: 368; columnNumber: 7; The element type "level" must be terminated by the matching end-tag "</level>".


Solution

Change it to the correct format and restart Artifactory.

<level value="debug"/>

Example 4

Format issue for the configuration file “binarystore.xml”:

For example, the "chain template" line is missing when adding/changing storage configuration:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<config version="2">
        <provider id="s3-storage-v3" type="s3-storage-v3">
                <endpoint>http://<s3_url></endpoint>
                <bucketName>test</bucketName>
                <identity>yourIdentity</identity>
                <credential>yourCredentials</credential>
         …
        </provider>
</config>

The console.log reports errors such as:

2023-10-07T07:12:30.661Z [jfrt ] [ERROR] [f68c9ceb15811728] [ctoryContextConfigListener:126] [art-init            ] - Application could not be initialized: Failed to load binary provider config. Reason only one chain definition is allowed in binarystore.xml
…
Failed to load binary provider config. Reason only one chain definition is allowed in binarystore.xml

Solution

Adding/modifying the configuration file with the correct parameter and restart Artifactory.

<config version="2">
        <chain template="s3-storage-v3"/>
        <provider id="s3-storage-v3" type="s3-storage-v3">
                <endpoint>http://<s3_url></endpoint>
                <bucketName>test</bucketName>
                <identity>yourIdentity</identity>
                <credential>yourCredentials</credential>
         …
         </provider>
</config>

Example 5

The system time between the Artifactory and the database is not synchronized:

The console.log will report a large number of the following errors:

2023-10-07T03:03:23.694Z [jfrou] [WARN ] [0f24f3779346dbdb] [join_executor.go:202          ] [main                ] [] - Cluster join: Error during cluster join: Error response from service registry, status code: 409; message: Requested Router Join for nodeId: bogon at: x.x.x.x conflicts (node id) with existing node: ServiceNodeImpl(serviceId=jfint@01hc44nd1keme8027d7y6a0zv4, nodeId=bogon, routerId=bogon, state=HEALTHY, created=1696651230258, lastUpdated=1696651458125, version=null, revision=null, ip=x.x.x.x, endpoints=[EndpointImpl(port=8082, secure=false, paths=[/integration/(.*), /com\.jfrog\.integration\..*])]); Retrying...
…

Solution

Use the command "ntpdate xxx" for time synchronization, or use the NTP server within your organization.

$ ntpdate pool.ntp.org

Then restart Artifactory.