Upgrading JFrog Artifactory to JFrog Platform – Q&A

Itamar Berman-Eshel
2020-01-16 12:20

 

What happens during the upgrade?

What’s under the hood in Artifactory version 7?

The new Artifactory version breaks down the legacy single service into several micro services, for more information, see this document.

In terms of internal communications between services, please see this document for a complete list of internal ports used by the different services.

To simplify things, the JFrog Platform uses only 2 external ports: 8081 for Artifactory REST APIs, 8082 for everything else (UI, and all other product’s APIs).

To use the REST api, the command needs to be executed against the JFrog Platform Deployment (JDP) IP with port 8082 and then append the service context:

<JFrog Base URL>:<Router Port>/<Service Context>/api/<Version>

I.e:

http://SERVER_HOSTNAME:8082/xray/api/v2

For Artifactory API, use port 8081:

http://SERVER_HOSTNAME:8081/artifactory/api/repositories

The upgrade

Other than on an archive installation, the installers pick up your current installation’s data and configuration files without any manual intervention. The archive upgrade is a semi-manual process where you are required to copy data and config files to the newly extracted archive and run the migration script – https://www.jfrog.com/confluence/display/JFROG/Upgrading+Artifactory#UpgradingArtifactory-LinuxArchiveUpgrade

This is what automatically happens when running the upgrade on all installation types:

  • Creating required directories – the new directory structure contains different folders for application files and configuration files, these are automatically created by the scripts.

  • Configuration migration – A system.yaml file is created at: $JFROG_HOME/artifactory/var/etc/system.yaml and configurations from different config files from your version 6.10.x installation (ha-node.properties, db.properties, xml and yaml files, default file) are consolidated into it.

  • Docker compose – Depending on your choices, a selected docker-compose.yaml will be available in the extracted folder. However, there are a few docker-compose templates in the directory templates.You can choose any template and copy it to the extracted folder as docker-compose.yaml.

What should I expect after the upgrade?

This Q&A section was created in order to assist users in finding known issues that might arise during the upgrade or installation of the JFrog platform.

Of course we would all like to be in a perfect world where everything is working flawlessly and new software is released without any bugs or issues, but that would be boring, right? And most of us would not have jobs 😉

As a general note that might be relevant to debugging most issues, there is a System Diagnostics script in each product’s app/bin/ folder that checks if the system is configured correctly to allow this service to run. We recommend first running it before the upgrade, in order to determine whether your system is ready, and when encountering issues before further investigating, this might indicate on what is the problem.

  • Will there be ability to rollback in case the migration fails?

    •  We encourage our users to create backups prior to starting the upgrade process since there is no option to roll back from an upgraded version.

  • Version compatibility between products, can I mix different versions?

    • Unified Experience will require an upgrade of all JFrog products.

  • My load balancer’s health check doesn’t work after the upgrade

    • The health check endpoint which should be used by the load balancer to check a node’s health has been changed from api/system/ping to /router/api/v1/system/ping

  • I have added new configuration parameters in my system.yaml but they are ignored

    • A yaml file is very sensitive to indentation, a parameter that is incorrectly indented might be ignored or corrupt the entire yaml configuration file. You should check that the file is correctly indented and/or compare your file with the product’s templates available at the same folder. As a recommendation and best practice, you can overwrite your values in the template and then rename it

  • I have stopped my application using ‘artifactoryctl stop’ but It fails when I try to run it again

    • Sometimes a microservice may not stop due to some reason, so starting the entire service again may fail due to a pid mismatch. Try to delete the pid file and restart the service

  • When trying to access the dashboard (JFMC) in the UI I get an error box saying connection refused to <JFrog Base URL> and port 9200 is specified

    • Port 9200 is used for ElasticSearch, check that the microservice is running.

  • Upgrading on a linux archive installer, the migrator script complains that it cannot find specific files

    • During the process, you are required to extract the archive and copy/move folders from your 6.10.x installation to the new filesystem structure, pay attention to the new filesystem structure and make sure the new JFROG_HOME/ filesystem matches the new structure – https://www.jfrog.com/confluence/display/JFROG/System+Directories

  • Upgrading an HA cluster, Artifactory complains that there is a problem with my master.key

    • When you want to add a node to your HA cluster, you are required to copy the master.key from the primary node and put it in the secondary node’s $JFROG_HOME/artifactory/var/etc/security/ folder.
      The master.key is sensitive to new line indentations, and editing it in a text editor might cause issues, we recommend using this command to create the file with the master key string copied from the primary node:
      ‘echo -n “<master key string>” > master.key’

  • In the Xray docker compose upgrade script, I answered the prompt with an incorrect answer and now i cannot change my choice even when re-running the config script

    • By default, all installation scripts should never overwrite content in the system.yaml, so if the yaml file already contains a configuration based on a previous run of the config script, it will not overwrite that configuration. We recommend removing the system.yaml before re-running.

Will I have downtime when upgrading Artifactory?

Yes, a downtime of a few minutes is required to upgrade JFrog Artifactory.

When upgrading an HA cluster, all nodes should be upgraded before running again.

 

Breaking changes in JFrog Platform

User Plugins

  • Description: As with any major version upgrade, user plugins that use private APIs may break due to changes in these APIs.
    Private APIs are subject to change at any point for any reason, which is why we don’t recommend their use.

  • Suggestion: Our recommendation is to test your plugins in a staging environment before upgrading the production environment. In general, we discourage the use of private Artifactory APIs in user plugins.

Public Context

  • Description: The current public conext, by default: <url>/artifactory had to be changed to accommodate the new platform’s architecture. The new UI now runs at <url>/ui. This means reverse proxy configurations will almost certainly need to be updated.

  • Suggestion: We suggest updating the reverse proxy configuration before upgrading. The context should be removed altogether and the reverse proxy should listen at the root (/).

Inter-product communication

  • Description: The Unified Experience has changed the way products communicate with each other. Previously all products used the HTTP(s) protocol to communicate. The products will now use gRPC which requires HTTP2 support. This could break certain setups if a reverse proxy, load balancer or other network device sits in between the products and it does not support HTTP2.

  • Why: This change was needed to accommodate the new architecture.

  • Suggestions: If anything sitting between the products does not support HTTP2, it can either be removed (the gRPC traffic will be encrypted by default) or changed to an equivalent that does support it.

Custom logback and Sumo Logic Integration

  • Description: New installation will override existing Artifactory (and Access) logback configuration – removing any customizations, including Sumo Logic configuration

  • Why: Complete overhaul of the logs structure, naming etc.

Artifactory and Xray 1:1 ratio

  • Description: When upgrading to the JFrog Platform, a 1:1 ratio between Xray to Artifactory is now required. If you have a single Xray instance connected to multiple Artifactory instances, before upgrading Artifactory and Xray, you will need to split your Xray instance to multiple instances to support this requirement. For the complete procedure, please see this document: https://www.jfrog.com/confluence/pages/viewpage.action?spaceKey=JFROG&title=Xray+and+Artifactory+One+to+One+Pairing
    Failing to follow this, and upgrading an Xray server which contains data from multiple Artifactory instances may lead to data loss!

Artifactory and Distribution 1:1 mapping

  • Description: The upgrade to the JFrog Platform requires a 1:1 mapping between a source Artifactory and a Distribution service. If you are creating and distributing release bundles from multiple source Artifactory instances, you now need to deploy a Distribution service in every JPD (JFrog Platform Deployment) that contains these source Artifactory instances. You would then register multiple Distribution services to one Mission Control.

Oracle database

  • Description: Working with an Oracle database requires a new driver and setup, as described here.

Xray MongoDB database

  • Xray’s MongoDB is no longer needed (except during the data migration). If you are upgrading to the new JFrog Platform, your data will be automatically migrated over to PostgreSQL as part of the upgrade process.

System Directories

  • Logs structure: has been updated from version 6.x, additional logs for new microservices have been added, log names have been changed to be aligned with all products and log output format has been changed to be aligned with all products.

    • Log file names are now prefixed by the service name and a dash. For example, the Artifactory log file name has been changed from application.log to artifactory-service.log.

    • Additional logs are now included for the new Router, Metadata, and Frontend.

    • There is an aggregated log called console.log which also emphasises the log source, log level and other information in color for better visibility.

System Configuration

  • Authentication Provider: You no longer configure an Artifactory as an Authentication Provider for Authentication as the new JFrog Platform architecture is based on a join-key method that automatically connects between the JPD and its services.

  • Accessing the UI: The JFrog Platform web UI is now accessed through port 8082 (For example, http://SERVER_HOSTNAME:8082/ui/). Accessing Artifactory directly for REST API and downloads is still possible through port 8081.

  • Reverse Proxy: The reverse proxy configuration should only be configured for the JFrog Platform, where previously it was configured for each JFrog Product separately. You will need to create a new reverse proxy configuration.

  • Load balancer health check endpoint: has been changed from api/system/ping to /router/api/v1/system/ping.

  • Replicator: All configurations have been moved from the replicator.yaml to the Artifactory system.yaml file.

Deprecated features in version 7

Artifactory License Control

  • Description: The Artifactory License Control is a feature in Artifactory that identifies licenses in certain package types and builds by reading metadata of the package/artifact. After the data is read, certain actions can be taken via a plugin or an external build tool based on the license information.

  • Alternative: Use JFrog Xray to get richer information and support for additional package types.

AJP

  • Description: The AJP connector is being removed and will no longer work. If you are using this, you will have to switch to HTTP.

  • Why: This change was needed to accommodate the new architecture.

  • Suggestions: AJP is used with Apache and Apache is still supported but you must update the reverse proxy configuration to access Artifactory via HTTP.

Stash Search Results

  • Description: The option to save your search results and go back to them later has been removed.

Mission Control scripts

  • Description: Scripts allow admins to express configuration as code. The DSL enables defining the state of the configuration and Mission Control orchestrated the REST APIs to achieve the configuration.

  • Suggestions: You can use the JFrog CLI or REST APIs to automate the configuration of  multiple instances.