Hugging Face Repository Structure

JFrog Artifactory Documentation

Products
JFrog Artifactory
Content Type
User Guide
ft:sourceType
Paligo

Hugging Face repositories are structured using two types of versions:

  • Integration version: these versions are similar to branches in a Git repository. In local repositories the only integration version is Main, and in remote repositories, the integration version is created according to the requested branch.

  • Release versions: these versions can be tags, commits, or revisions.

Starting from Artifactory version 7.77, the Hugging Face repository structure for local repositories is as follows:

 ├── hf-local
   ├── models
    ├── org (optional)
        ├── model-name 
            ├── main     
              ├── timestamp
                    ├── .jfrog_huggingface_model_info.json
                    ├── file1 
                    ├── file2
              ├── timestamp 
                    ├── .jfrog_huggingface_model_info.json
                    ├── file1    
                    ├── file2
              ├── timestamp (server)
                    ├── .jfrog_huggingface_model_info.json
                    ├── file1    
                    ├── file2
            ├── v1 	
                ├── timestamp
                    ├── .jfrog_huggingface_model_info.json
                    ├── file1    
                    ├── file2

Note the following:

  • When resolving a model without specifying a revision, the latest model version will be saved with the timestamp under the ‘main’ branch in the model name folder. Previous versions are retrievable using the SHA1 value, which is stored as the property huggingfaceml.generated.revision.sha1 in the .jfrog_huggingface_model_info.json file.

  • When deploying a model with a specific version, it will be saved with a timestamp under that version name in the model name folder. If the same model has been deployed already with the same version number, the new deployment will overwrite it, and the previous deployment will not be retrievable using the Artifactory-generated SHA1 value.

    Note

    For Artifactory to overwrite the model, go into Administration > User Management > Permissions, and verify that the Delete/Overwrite permission is enabled. Otherwise, when trying to deploy a model that has been deployed before with the same version number, Artifactory will return a 409 error.

Starting from Artifactory version 7.77, the repository structure for remote repositories is as follows:

├── hf-remote
        ├── models
    ├── org (optional)
        ├── model-name
            ├── main 
		   ├── .latest_huggingface_model_info.json
                ├── timestamp 1
			├── .jfrog_huggingface_model_info.json
                    ├── file1   
                    ├── file2
                ├── timestamp 2
			├── .jfrog_huggingface_model_info.json
                    ├── file1    
                    ├── file2
                ├── timestamp 3
                    ├── .jfrog_huggingface_model_info.json
                    ├── file1    
                    ├── file2

            ├── v1 
   ├── .latest_huggingface_model_info.json
                ├── timestamp 1
			├── .jfrog_huggingface_model_info.json
                    ├── file1   
                    ├── file2
                ├── timestamp 2
                    ├── .jfrog_huggingface_model_info.json
                    ├── file1  
                    ├── file2
            ├── original hash
                ├── .latest_huggingface_model_info.json
                ├── timestamp
			├── .jfrog_huggingface_model_info.json
                    ├── file1    
                    ├── file2

Note the following:

  • When resolving a model without specifying a revision, the latest model version will be saved with the timestamp under the ‘main’ branch in the model name folder. Previous versions are retrievable using the SHA1 value, which is stored as the property

    huggingfaceml.generated.revision.sha1 on the .jfrog_huggingface_model_info.json file.

  • When resolving a model with a specific version using a commit hash or tag, it will be saved with a timestamp under the requested revision in the model name folder.