How to troubleshoot NPM problems

Subject 

Comparing common NPM queries with a reference can be used to spot failing API calls.

Affected Versions

4.0.2 – Latest

Description

The NPM client is used by many application pipelines to pull javascript dependencies for web UI functionality. It has some API calls that are unique and may cause errors if not properly handled by Artifactory.

When troubleshooting NPM errors it is best to compare what is supposed to happen against what is currently happening. You can eliminate a great deal of noise by simulating each step the client takes, and examining what HTTP return code is sent by Artifactory.

In general, these return codes indicate where to look next:
403 and 401 – Authentication issue, examine authorization settings
404 – Resource not found, check repository for resource
400 – Incorrect request, use curl to get reason phrase
50X – Server problem, examine artifactory.log

Resolution

Using curl
The libcurl terminal application can make basic REST API commands such as GET or PUT options. The Artifactory request logs display the exact API commands the Docker client uses.

You can use the examples below, or substitute the one that is failing in the Artifactory request.log file. Doing so should yield further information on the problem, and possible next steps.

Basic usage (with #comments):

curl -uadmin #Artifactory-username -vvv #verbosy -k #ignore-insecure-SSL -XGET #GET-request http://localhost:8081/artifactory/api/system/ping
 

Curl uses GET requests by default. Other commands like "PUT" require the –data field, along with the (Usually JSON) data payload. An example PUT command:

curl -uadmin:password -XPUT -H"Content-type: Application/json" –data '{"name":"test-group"}' http://localhost:8081/artifactory/api/security/groups/test-group

NPM Login
When a user runs an NPM login, the NPM client will prompt them for their information:

jfrog@jfrog:~/development/npm$ npm login
Username: admin
Password: 
Email: (this IS public) admin@admin.com

On the backend NPM will post this data to Artifactory for future use:

20180523134454|10365|REQUEST|127.0.0.1|anonymous|PUT|/api/npm/npm/-/user/org.couchdb.user:admin|HTTP/1.1|201|153

NPM Install (Non-scoped):
When a user runs an NPM install, the NPM client gets metadata then returns the package:

npm install foobar

1. The NPM client requests metadata on the package:

20180523134502|2498|REQUEST|127.0.0.1|admin|GET|/api/npm/npm/foobar|HTTP/1.1|200|0

2. The metadata contains the download link for the tar.gz file:

20180523134503|386|REQUEST|127.0.0.1|admin|GET|/api/npm/npm/foobar/-/foobar-1.1.0.tgz|HTTP/1.1|200|10240

And dependency files as well:

20180523134506|26|REQUEST|127.0.0.1|admin|GET|/api/npm/npm/busybox/-/busybox-2017.3.22.tgz|HTTP/1.1|200|518376

The metadata is pulled from a “package.json” file found in each NPM .tgz file:

{
    "name": "@types/webpack",
    "version": "3.8.5",
    "description": "TypeScript definitions for webpack",
    "license": "MIT",
    "contributors": [
[…]
"main": "",
    "repository": {
        "type": "git",
        "url": "https://www.github.com/DefinitelyTyped/DefinitelyTyped.git"
    },
"scripts": {},
    "dependencies": {
        "@types/tapable": "*",
        "@types/uglify-js": "*",
        "@types/node": "*"
    },
    "typesPublisherContentHash": "9cb451892d84836f57c4925bdc1308f145e527eed87c3610683ada4330d844ca",
    "typeScriptVersion": "2.0"
}

NPM install (Scoped Packages)

npm install @angular/cli

NPM install for scoped packages (@angular/router) uses an encoded slash for the “/” between the scope and the package name. This should be displayed as “%2f” in the Artifactory request logs:

20180731002214|166|REQUEST|127.0.0.1|admin|GET|/api/npm/npm/@angular%2fcli|HTTP/1.1|200|0
20180731002214|190|REQUEST|127.0.0.1|admin|GET|/api/npm/npm/@angular/cli/-/cli-6.1.1.tgz|HTTP/1.1|200|128696

There are 2 places where the encoded slash may fail. 

One is on the Artifactory JVM options (“-Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true”), which has been enabled by default since Artifactory 4.4.3.

The other location is on reverse proxies. Artifactory’s generated reverse proxy settings have the correct configuration to leave the slash as unencoded:

 ##The “/” in the proxy_pass line forces Nginx / Apache to not decode encoded characters
proxy_pass          http://localhost:8081/artifactory/;

NPM Publish
To publish NPM packages to Artifactory, you need to configure your package.json to point to your Artifactory NPM repository:

{
  "name": "test-package",
  "version": "0.0.16",
  "description": "hahahahah",
  "main": "index.js",
  "scripts": {
    "test": "echo "Error: no test specified" && exit 1"
  },
  "author": "Pat",
  "license": "ISC",
  "publishConfig": {
    "registry": "http://localhost:8081/artifactory/api/npm/npm/"
  },
  "dependencies": {
    "angular": "^1.6.4",
    "busybox": "^2017.3.22",
    "clone": "^2.1.1",
    "uglify-js": "^2.7.5"
  },
  "devDependencies": {},
  "maintainers" : [
    {"name":"jimmy", "email":"jim@jimmy.com"},
    {"name":"Frank","email":"frank@west.com"}]
}

After doing so, you can run “npm publish” to deploy a .tar.gz file to the local repository:

npm publish

[request.log]
20180524084923|96|REQUEST|127.0.0.1|admin|PUT|/api/npm/npm/test-package|HTTP/1.1|201|1695

Dependency Rewrites
One common setting for NPM is the “Enable Dependency Rewrite” option for Virtual Repositories. This allows Artifactory to cache GitHub metadata associated with NPM packages, and serve the content locally.

When a remote repository is set as the dependency cache, an “_external” folder is created that stores GitHub .tar.gz files:
_external folder contains *.tar.gz files from github
Sometimes Artifactory cannot reach GitHub. If you see such occurrences in the logs, consider disabling Dependency Rewrites and see if it resolves the problem.