ARTIFACTORY: Smart Remote Repository Configuration for PyTorch PyPI Registry in Artifactory

Introduction

This article outlines the steps required to configure a Smart PyPI Remote Repository for accessing the PyTorch PyPI Registry through JFrog Artifactory.

A Smart Remote repository is a remote repository that proxies a local, remote, or Federated repository from another instance of Artifactory or an Edge node.

First, on the main Artifactory instance, create a PyPI remote repository that points to the PyTorch PyPI registry URL. This repository acts as an upstream source, allowing Artifactory to proxy and cache packages retrieved from the PyTorch registry.

Once the upstream PyPI remote repository is successfully configured and validated, we then configure a Smart Remote Repository on the target Artifactory instance. This setup allows the target instance to fetch PyTorch PyPI packages through the Smart Remote repository.



Step-by-Step Instructions

Set up a PyPI remote repository with the PyTorch registry URL as the upstream source. The configuration parameters to be defined are as follows:

URL: https://download-r2.pytorch.org

Registry URL: https://download.pytorch.org

Registry Suffix: whl

Validate that packages can be successfully installed from the configured PyPI remote repository.

#pip install --no-cache-dir --index-url
https://<username>:<token>@jpd.local/artifactory/api/pypi/test-pytorch-remote/simple --no-deps torch
Looking in indexes:
https://<username>:****@jpd.local/artifactory/api/pypi/test-pytorch-remote/simple
Collecting torch
  Downloading torch-2.13.0-cp314-cp314-macosx_14_0_arm64.whl (111.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 111.2/111.2 MB 7.2 MB/s  0:00:15
Installing collected packages: torch
Successfully installed torch-2.13.0

To configure the Smart Remote Repository, create a PyPI remote repository on the target Artifactory instance that points to the main Artifactory instance URL, using the configuration parameters outlined below:

URL: https://jpd.local/artifactory/api/pypi/test-pytorch-remote

Registry URL: https://jpd.local/artifactory/api/pypi/test-pytorch-remote

Registry Suffix: Simple

Ensure that the credentials (username and password) for the source Artifactory instance with the specified URL are provided and “test connection” is successful.

Verify that packages can be successfully installed using the Smart Remote Repository

Textproto

# pip install --no-cache-dir --index-url
https://<username>:<token>@jpd2.local/artifactory/api/pypi/test-pytorch-smart-remote/simple
--no-deps torch
Looking in indexes:
https://<username>:****@jpd2.local/artifactory/api/pypi/test-pytorch-smart-remote/simple
Collecting torch
  Downloading torch-2.13.0-cp314-cp314-macosx_14_0_arm64.whl (111.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 111.2/111.2 MB 9.5 MB/s  0:00:11
Installing collected packages: torch
Successfully installed torch-2.13.0