Artifactory supports resolving specific models, datasets, and related files using libraries like Transformers and Diffusers, among others.
Note
Resolving Hugging Face models using libraries is only supported from Artifactory version 7.77 and above, Hugging Face client version 0.19.0 and above, and with the HF_HUB_ETAG_TIMEOUT parameter enabled. Otherwise, use snapshot download to resolve whole model repositories.
Model Example
In the following example, we will use the Transformers library to download a model called Llama-2-7b-chat-hf using artifactory. Transformers is the library supported by this model: note that other models might have different library options.
To resolve the Llama-2-7b-chat-hf model from an Artifactory repository using Transformers:
Go to the model page on Hugging Face Hub, click Use this model on the top right side of the screen, and select Transformers from the drop-down menu
Click Copy next to the command you want to use
Run the command in your code to resolve the model
Dataset Example
In the following example, we will use the Datasets library to download a dataset called wikipedia using artifactory. Note that Datasets is the only library used by Artifactory to download datasets.
To resolve the wikipedia library from an Artifactory repository using Datasets:
Go to the dataset page on Hugging Face Hub, click </> Use this dataset on the top right side of the screen, and select Datasets from the drop-down menu
Click Copy next to the command you want to use