ARTIFACTORY: Enabling Fast Dependency Resolution for PyPI (PEP 658)

Note:

The steps described in this article apply only to Self-Hosted (On-Premises) customers. If you are a SaaS (Cloud) customer, you can open a Support ticket with JFrog to have this configured.



Overview: What is data-dist-info-metadata?

When pip installs a package, it traditionally has to download the entire distribution file (a .whl or .tar.gz) just to read the metadata inside it to determine dependencies. For large packages, this wastes significant time and bandwidth.

The data-dist-info-metadata attribute (defined in PEP 658) allows Artifactory to serve a tiny sidecar .metadata file (usually ~1KB) independently of the main package. This enables pip to perform "Fast Dependency Resolution", it reads the tiny file to plan the installation and only downloads the "heavy" package once it is 100% sure it's the correct version.



Implementation in Artifactory

To enable this capability in your Artifactory instance, you must toggle a system property.



1. Enable the Property

Navigate to $JFROG_HOME/artifactory/var/etc/artifactory.

Edit the artifactory.system.properties file.

```
Add the following property:
```

artifactory.pypi.add.data.dist.info.metadata.attribute=true

Restart Artifactory for the changes to take effect.



Verification: How to Check It Works

Step 1: Verify the Index Attribute

Execute a cURL against your PyPI repository's simple index. You should see the attribute and its SHA-256 hash in the HTML link:

curl -u <user>:<password> http://<URL>/artifactory/api/pypi/<repo>/simple/<package>/ |
grep data-dist-info-metadata



Expected Output:

The link will contain data-dist-info-metadata="sha256=...". This hash acts as a digital fingerprint for the metadata file.



Step 2: Inspect the Metadata Content

You can manually download the tiny metadata file by appending .metadata to the actual package path:

curl -u <user>:<password> http://<URL>/artifactory/<repo>/<path-to-file>/package.whl.metadata

This should return plain text containing the package's dependencies (e.g., Requires-Dist: ...).



Step 3: Verify with Pip

Run an installation using pip (v22.2+) with verbose logging. Pip will explicitly state it is using the metadata file instead of the full wheel:

pip install <package> --index-url http://<URL>/.../simple --no-cache-dir -vvv



Look for this line in the logs:

Obtaining dependency information for <package> from http://.../<package>.whl.metadata

When enabled, Artifactory includes the data-dist-info-metadata attribute in PyPI package index entries (HTML and JSON). The property defaults to false.



Local repositories

Packages uploaded while the property is enabled are indexed with the attribute.

After changing the property (enabled → disabled or disabled → enabled), reindex the repository so the package index reflects the new setting.



Remote repositories

Artifacts cached while the property is enabled are served from cache with the attribute.

Artifacts cached while the property is disabled are served from cache without the attribute.

After changing the property (enabled → disabled or disabled → enabled), zap the remote repository cache so subsequent downloads pick up the new setting.



Summary

Reduced Bandwidth: Saves megabytes (or gigabytes) of traffic in CI/CD pipelines.

Faster Resolution: pip can decide if a package is compatible in milliseconds.

Integrity: The SHA-256 hash ensures the metadata hasn't been tampered with before pip reads it.

Note:

The steps described in this article apply only to Self-Hosted (On-Premises) customers. If you are a SaaS (Cloud) customer, you can open a Support ticket with JFrog to have this configured.