Going Beyond Exclude Patterns: Safe Repositories With Priority Resolution

Going Beyond Exclude Patterns: Safe Repositories With Priority Resolution

You probably remember the Namespace Shadowing a.k.a. “Dependency Confusion” attack that was in the news a couple of weeks ago. I blogged back then about the Exclude Patterns feature of JFrog Artifactory which we’ve had forever and was always intended to protect you against those kinds of attacks.

Since the topic was in the news, it brought the exclude patterns feature a lot of attention, obviously making your builds more secure (you use it now, right?!), but also highlighting some of the drawbacks of the approach.

We heard your feedback loud and clear. While Exclude Patterns work and are easy to implement If your internal libraries do not conform to a small set of common naming patterns you’ll have to set up many exclusion rules which is not a pleasant experience and may also have an impact on performance. You wanted control on a repository level. You’ve got it. Since Artifactory 7.16.3 you can now flag a Local or Remote repository as a “Priority Resolution”.

TL/DR: Flip this checkbox on repositories containing your private packages, and Artifactory won’t look for new versions of those packages out there.

For those of you, who love the technical details, here is how it works:

When a package manager requests a dependency, it will (in the general case) first ask for the metadata (list of all available versions), then make a decision which version it needs (based on your dependency declaration), and then request a specific file.

When your package manager is configured to work with a virtual repository in Artifactory, the repository points to a set of local and remote repositories. Artifactory will prepare the metadata for the client, parsing all the files in all the repositories in order, based on two rules: (1) The resolution order you set up in the settings of the virtual repository, and (2) local repositories always queried before the remote ones.

Let’s consider a couple of scenarios to demonstrate. Here’s the setup:

We have a set of local repositories for the promotion pipeline, and a remote repository that proxies a central registry, all included in one virtual repository. The order of resolution defined in the virtual repository settings is as follows:

  • Local prod repository
  • Local staging repository
  • Local dev repository
  • Remote repository

Even if you put a remote repository higher than a local one, don’t worry! Artifactory will take care of rearranging the order, always resolving from local repositories first. Let’s run some scenarios on this setup.

Scenario 1: Listing versions of an external dependency

  1. A user defines dependency on `external-package` version ~1.0.0 (any version backwards compatible with 1.0.0, i.e. everything up to, but not including, 2.0.0)
  2. A client asks for metadata on package ‘external-package’ in a range between 1.0.0 up to (but not including) 2.0.0.
  3. Artifactory checks all three local repositories, can’t find existing metadata for external-package, as it doesn’t exist there.
  4. Artifactory checks the remote repository, where it finds metadata for `external-package` (e.g. versions 1.1.0 and 1.2.0).
  5. No metadata merging needed, central registry metadata returned as-is containing both found versions.

Going Beyond Exclude Patterns

Scenario 2: Listing versions of an internal dependency with different versions in different repositories

  1. A user defines dependency on `external-package` version ~1.0.0 (any version backwards compatible with 1.0.0, i.e. everything up to, but not including, 2.0.0)
  2. A client asks for metadata on package ‘internal-package’ in a range between 1.0.0 up to (but not including) 2.0.0.
  3. Artifactory checks all three local repositories, finds different metadata listings in all three (let’s say prod contains versions 1.1.0, 1.2.0, and 1.3.0, staging contains versions 1.4.0 and 1.5.0, and dev contains version 1.6.0).
  4. Artifactory checks the remote repository, no `internal-package` found there, either cached or in central registry.
  5. Artifactory merges the metadata, producing a list of versions 1.1.0 to 1.6.0 as all of them can be fetched from our virtual repository.

Going Beyond Exclude Patterns

Scenario 3: Namespace shadowing attack

  1. The attacker uploads a malicious version of our `internal-package` as version 1.7.0 into an external repository.
  2. A user defines dependency on `external-package` version ~1.0.0 (any version backwards compatible with 1.0.0, i.e. everything up to 2.0.0)
  3. A client asks for metadata on package ‘internal-package’ in a range between 1.0.0 to 2.0.0.
  4. Artifactory checks all three local repositories, finds different metadata listings in all three (let’s say prod contains versions 1.1.0, 1.2.0, and 1.3.0, staging contains versions 1.4.0 and 1.5.0, and dev contains version 1.6.0).
  5. Since no exclude patterns are defined on our remote repository (tisk-tisk) Artifactory checks it, and finds the metadata for `internal-package`, listing the malicious version 1.7.0.
  6. Artifactory merges the metadata, producing a list of versions 1.1.0 to 1.7.0 as all of them can be fetched from our virtual repository.
  7. The client resolves the latest version (1.7.0) and gets hacked.

Going Beyond Exclude Patterns

Marking the repositories with the Priority Resolution flag takes precedence over the resolution order when resolving virtual repositories. Setting repositories with priority will cause metadata to be merged only from repositories set with this field, effectively canceling the step (e) in our third example (effectively converting a bad scenario 3 into a good scenario 2), so the returned metadata will only list 6 versions, so the client won’t know about the malicious version 1.0.7, and won’t ask for it.

Priority Resolution flag

Obviously, even with repositories set for Priority Resolution, if the artifact is not found in any of the local repositories (as in our first example), the lookup in the other repositories (not set for Priority Resolution) will occur (finding the `external-package` in the central registry).

While most of the time you’ll set a local repository for Priority Resolution, sometimes you have a remote repository which proxies a trusted Artifactory instance (another development site, maybe?), so Priority Resolution can be set on remote repositories as well.

So, go ahead, upgrade to the latest version of JFrog Artifactory and mark the repositories with your internal dependencies as “Priority Resolution” to be protected against Namespace Shadowing attacks. We still highly recommend scoping your internal dependencies with namespaces (or at least naming conventions) and using the exclude patterns with those.