ARTIFACTORY: Troubleshooting Guide - Installation and Upgrade

This guide provides solutions to common issues encountered during the installation and upgrade of Artifactory.

Issue 1: Missing Join Key During Installation

Issue Description: Artifactory might fail to start with the "Cluster join: Join key is missing" error repeatedly appearing. This issue occurred despite attempting to add the join key via system.yaml or join.key files.

Related Error Messages:

Cluster join: Join key is missing

Steps to Resolve:

If the join key is missing, Artifactory will automatically generate a new one. However, if the join key already exists, you will receive a join key mismatch error. Therefore, a missing join key most likely indicates a database connection problem.

Verify Database Connections: Test connections to make sure that you are able to make connections to the database.
Review Database Connection settings: Revisit database configuration settings here. For an external database, make sure the schema is created, and the database is configured to allow access. Follow this common debug procedure.


Issue 2: Artifactory Fails to Start Due to Insufficient Resources

Issue Description: Artifactory may fail to start, often entering a restart loop (e.g., in a Kubernetes environment), or the service may time out. There is typically no clear error message provided.

Related Error Messages:

[jfac ] [WARN ] [7bffdc6b0a5ba7e8] [o.j.c.ExecutionUtils:289 ] [jf-common-pool-3 ] - Retry 1 Elapsed 490 millis
Pods stuck in CrashLoopBackOff or OOMKilled state.

Steps to Resolve:

Insufficient resources can prolong the startup of Artifactory microservices. When critical microservices like jfrt/jfac experience extended startup times, repeated connection retry messages will appear without clear error indicators.

Review Resource Allocation: For optimal resource settings, consult the installer helper or the sizing documents. Note that for Kubernetes deployments of smaller sizes and above, resource issues should not hinder startup.
Review Database Resources for HA Clusters:
When additional nodes in High Availability (HA) clusters experience startup loops, it's crucial to examine your database configuration, specifically the maximum allowed connections.

Here's a useful tip for calculating the total number of connections:
Total # of connections = (number of nodes) * ((artifactory.database.maxOpenConnections * 2) + access.database.maxOpenConnections + metadata.database.maxOpenConnections) + 50;

For more detailed information, refer to Database Tuning for Heavy Loads in Artifactory.

Issue 3: Corrupted logback.xml prevents Artifactory Startup

Issue Description: A misconfigured or corrupted logback.xml file can cause the Artifactory service to fail during startup, as it cannot initialize its logging framework.

Related Error Messages:

If Artifactory ping fails without a clear error message in the artifactory-service.log, it might indicate corruption. You can check the Artifactory ping status using the command: $ curl https://<JFrog_Platform_URL>/artifactory/api/system/ping.

Steps to Resolve:

Validate logback.xml: Check the logback.xml file for any syntax errors or incorrect configurations.
Restore from Backup: If possible, restore the logback.xml file from a known-good backup or fresh install.


Issue 4: Incorrect JDBC Driver Causes Connection Failure

Issue Description: Artifactory fails to connect to its database due to an incorrect, missing, or incompatible JDBC driver.

Related Error Messages:

java.lang.ClassNotFoundException: Didn't find class "java.lang.management.ManagementFactory"
Caused by: java.lang.ClassNotFoundException: com.mysql.cj.jdbc.Driver
Failed to load driver class org.postgresql.Driver in either of HikariConfig class loader or Thread context classloader

A "class not found" error typically indicates an invalid database.

Steps to Resolve:

Verify Driver Location: Ensure the correct JDBC driver JAR file is placed in [ARTIFACTORY_HOME]/var/bootstrap/artifactory/tomcat/lib and [ARTIFACTORY_HOME]/app/artifactory/tomcat/lib, and remove the incorrect driver from both locations
Put the correct JDBC driver jar under
$JFROG_HOME/artifactory/var/bootstrap/artifactory/tomcat/lib
And make sure it’s owned by artifactory user
Check Driver Version: Confirm that the JDBC driver version is compatible with both your database version and the Artifactory version. You can follow this to perform a validation test.
Review system.yaml: Check the shared.database.driver and shared.database.url parameters in the system.yaml file to ensure they are configured correctly for your database type.


Issue 5: Lack of License Keys for New Nodes

Issue Description: When adding new nodes to an existing High Availability cluster, the new nodes may fail to join if there are not enough available license keys in the license bucket.

Related Error Messages:

[jfrt ] [WARN ] [337091a0354ae897] [o.a.r.f.LicenseRestFilter:74 ] [http-nio-8081-exec-8] - License is not installed

Steps to Resolve:

Add Licenses: In the Artifactory UI, navigate to Administration > Licenses and add the license keys for the new nodes. Refer to License Management for more information.
Restart New Nodes: After adding the licenses, restart the new Artifactory nodes so they can acquire a license and join the cluster.


Issue 6: Bad Padding Exception When Adding New Nodes

Issue Description: Adding a new node to an HA cluster fails with a "Given final block not properly padded" error. This is often caused by incorrectly copying an already-encrypted join.key from an existing node instead of using the original master.key.

Related Error Messages:

javax.crypto.BadPaddingException: Given final block not properly padded. Such issues can arise if a bad key is used during decryption.

Steps to Resolve:

Use the Master Key: When setting up a new node, ensure you copy the master.key from the primary node to the new node's [ARTIFACTORY_HOME]/var/etc/security/ directory. Do not copy the join.key.
Restart New Node: After placing the correct master.key file, start the new Artifactory node. It will use the master key to securely communicate with the primary node and generate its own valid join.key.


Issue 7: Large Plugin Files Delay Artifactory Startup

Issue Description: The presence of large or numerous custom Groovy plugins in the [ARTIFACTORY_HOME]/var/etc/artifactory/plugins/ directory can significantly delay the startup of the Artifactory service and causing it fail to start up. 

Related Error Messages:

Long pauses in the logs after the "Loading script from..." messages.
Router readiness checks may fail with connect: connection refused if the Artifactory (jfrt )service takes too long to start.

Steps to Resolve:

Review Plugins: Audit the plugins directory. Identify and remove any large, outdated, or unnecessary plugin files (.groovy, .json).
Test in a Staging Environment: Before deploying new plugins to production, test their impact on startup time and performance in a non-production environment.
Optimize Plugins: If a custom plugin is necessary, review its code for performance bottlenecks or inefficient operations.


Issue 8: "No valid installed license found" After Upgrade

Issue Description: After upgrading Artifactory, the system fails to start with an error indicating that no valid license was found.

Related Error Messages:

No valid installed license found

Steps to Resolve:

Verify License Compatibility: Ensure that your license is valid for the version of Artifactory you are upgrading to.
Re-apply License: If the license is valid, try re-applying it through the Artifactory UI or by using the appropriate REST API endpoint.


Issue 9: Access Migration Failure During Upgrade

Issue Description: The upgrade process fails during the database migration phase, specifically with an error related to "PermissionsV2DataMigration". This often indicates an issue with the underlying database schema or data that is incompatible with the new version.

Related Error Messages:

Could not initialize database: org.flywaydb.core.api.FlywayException: Detected failed migration to version 7.82.0.1 (PermissionsV2DataMigration)

Steps to Resolve:

Backup Database: Before attempting any manual intervention, ensure you have a complete and validated backup of your Artifactory database.
Consult Knowledge Base: This is a known issue. Refer to the official JFrog Known Issues page for the"PermissionsV2DataMigration" issue.
Contact Support: Due to the complexity of database migrations, it is highly recommended to open a support ticket with JFrog for assistance. Provide the full error logs and details about your environment.


Issue 10: "Found an init lock in the DB" Error During HA Upgrade

Issue Description: During an upgrade of a High Availability cluster, one or more nodes fail to start, reporting that an "init lock" was found in the database, but the owner is not the current node. This typically happens when a previous upgrade attempt failed, leaving a stale lock in the database.

Related Error Messages:

Found an init lock in the DB, but the owner is not the current node

Steps to Resolve:

Identify the Lock Owner: Check the Artifactory logs to identify which node currently holds the database lock.
Clear the Lock: Refer to the official JFrog Knowledge Base for the recommended procedure to safely clear the init lock from the database. This usually involves running a specific SQL query.
Restart Upgrade: Once the lock is cleared, you can safely restart the upgrade process on the failed nodes.