Introduction
When Conan builds a missing package binary, the package recipe may download its source archive directly from GitHub or another external host. This request is made by the Conan client and is separate from recipe and binary downloads through an Artifactory Conan repository.
In a network that blocks or intercepts external traffic, the Conan remote can work while the build still fails in the recipe's source() method. Conan backup sources solve this for eligible recipes by storing the source archives in an Artifactory Generic repository.
Issue
The Conan client connects to Artifactory and resolves the dependency graph, but a package that must be built from source fails while downloading from an external site:
ERROR: fmt/12.1.0: Error in source() method, line 86
get(self, **self.conan_data["sources"][self.version], strip_root=True)
ConanException: Error downloading file
https://github.com/fmtlib/fmt/releases/download/12.1.0/fmt-12.1.0.zip:
'HTTPSConnectionPool(host='github.com', port=443): Max retries exceeded with url:
/fmtlib/fmt/releases/download/12.1.0/fmt-12.1.0.zip (Caused by SSLError(SSLCertVerificationError(1,
'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate
(_ssl.c:1010)')))'Allowing access to a Conan repository such as Conan Center does not cover this request. Artifactory serves the Conan recipe and any available binary packages, while the recipe's source() method controls where source archives are downloaded from when Conan builds a binary.
How Conan backup sources work
Conan backup sources provide a mirror for source files downloaded by conan.tools.files.get() or conan.tools.files.download(). The recipe must supply the file's SHA-256 checksum.
The workflow uses two clients:
-
An internet-connected seed client downloads the original sources, saves them in Conan's local source-backup cache, and uploads them to Artifactory.
-
A restricted client uses only the Artifactory backup URL and does not fall back to the original external URL.
Each source archive is stored in the Generic repository as:
Prerequisites
-
An Artifactory administrator who can create a local Generic repository
-
An internet-connected seed machine
-
A restricted build machine that can reach Artifactory
-
Conan 2 installed on both machines
-
If the Generic repository requires authentication, configure access for the backup URL before continuing. See Conan's [source_credentials.json documentation].
Create the Artifactory repository
-
In the Artifactory Administration module, create a local repository with the Generic package type.
-
Give the repository a descriptive name, for example:
conan-backup-sources
The repository URL used in the Conan configuration is:
https://<JFROG_BASE_URL>/artifactory/conan-backup-sources/
The backup repository remains separate from the Conan local, remote, and virtual repositories. Do not add the Generic repository to a Conan virtual repository.
Configure the internet-connected client
1. Configure the backup URLs
Add the following settings to ~/.conan2/global.conf:
core.sources:download_urls=["https://<JFROG_BASE_URL>/artifactory/conan-backup-sources/", "origin"]
core.sources:upload_url=https://<JFROG_BASE_URL>/artifactory/conan-backup-sources/
The backup URL appears first so Conan checks Artifactory before the original source URL. The "origin" entry allows the seed client to download a missing source from its original location.
2. Build the missing packages
Run the normal install from the project directory:
jf conan install . --build=missing
The required dependency must build from source for Conan to populate the source-backup cache. If Conan downloads an existing binary, its source() method does not run and that source archive is not added to the cache.
3. Upload the cached sources
Upload all source backups currently present in the Conan cache:
conan cache backup-upload
conan cache backup-upload uploads every cached source backup. It cannot be limited to a single package reference.
4. Verify the upload
Open the conan-backup-sources repository in Artifactory. Confirm that each backed-up source has both a SHA-256-named blob and a matching .json file.
SHA-256 source blobs and metadata files in the Artifactory Generic repository
In the reproduced fmt/12.1.0 case, the source archive used this SHA-256 value:
695fd197fa5aff8fc67b5f2bbc110490a875cdf7a41686ac8512fb480fa8ada7
Configure the restricted client
1. Remove the internet fallback
On the restricted machine, add only the Artifactory backup URL to ~/.conan2/global.conf:
core.sources:download_urls=["https://<JFROG_BASE_URL>/artifactory/conan-backup-sources/"]
Do not include "origin". Without it, Conan cannot fall back to GitHub or another external source host.
The restricted client does not need core.sources:upload_url.
2. Test with a clean Conan cache
Use a new Conan home or a clean cache so the test does not reuse a previously downloaded source or binary. Run an install that requires the package to be built:
jf conan install . --build=missing
When Conan retrieves an eligible source from Artifactory, the output includes a message in this form:
Sources for [...]
found in remote backup https://<JFROG_BASE_URL>/artifactory/conan-backup-sources
The build should proceed without a request to the original GitHub or vendor URL.
Troubleshooting
Conan still connects to GitHub
Check the following:
-
The restricted client's effective core.sources:download_urls does not include "origin".
-
The seed machine uploaded the expected SHA-256 blob and .json file.
-
The build process uses the expected Conan home and global.conf.
-
The recipe uses get() or download() and supplies a sha256 value.
Display the effective backup-source configuration with:
conan config show "core.sources:*"
The backup repository is unreachable
Conan does not skip an unreachable backup URL and continue to the next entry. Verify the Artifactory URL and network connectivity.
The source is not uploaded
Backup sources do not cover every source-acquisition method. They do not mirror:
-
Git.clone()
-
Downloads performed through self.run()
-
Custom Python download implementations
-
get() or download() calls without a SHA-256 value
These recipes must be changed to use an eligible, checksum-verified download method or handled through a separate mirroring process.
Notes
-
Keep the trailing slash on the Artifactory backup URL.
-
The Generic repository stores raw source archives. Conan packages remain in repositories with the Conan package type.
-
The seed process must be repeated when a new source checksum is introduced and is not already present in the backup repository.
References