SageMaker training jobs require launching a SageMaker compatible Docker image. For this example, the prepared training script will be wrapped within a Docker image that is set up with the expected directory structure and environment variables to run in the SageMaker environment. Refer to the Dockerfile in the train directory available here.
Note the first several lines of the Dockerfile — to retrieve the base image required, the FROM statement is set up to proxy a request to Docker Hub via the virtual Docker repository set up in Artifactory.
Another item to note in this file is the use of a Docker build secret to supply the pip configuration necessary to resolve all of the required Python package dependencies needed to fulfill the requirements for the training script (and later for inference).
Both the requirements.txt file and the pip.conf file are available in the git repository.
Update the pip.conf file as required with the credentials needed for the virtual PyPi repository in Artifactory, then build, tag and upload the custom Docker image to Artifactory. The following is an example command to accomplish this using the Dockerfile in the github repo. Remember to modify the Artifactory host name, the image name/tag, and the name of the virtual PyPi repository as needed for your environment.
Note the format of the Docker tag. This format is required to identify the Artifactory registry and repository where the image will be pushed.
Note the first several lines of the Dockerfile — to retrieve the base image required, the FROM statement is set up to proxy a request to Docker Hub via the virtual Docker repository set up in Artifactory.
# Artifactory server/registry and repo for resolving Docker images
ARG ARTIFACTORY_DOCKER_REGISTRY
ARG ARTIFACTORY_DOCKER_REPO
# Download an open source TensorFlow Docker image to use as the Base Image
FROM ${ARTIFACTORY_DOCKER_REGISTRY}/${ARTIFACTORY_DOCKER_REPO}/tensorflow/tensorflow:latest-gpu-jupyter Another item to note in this file is the use of a Docker build secret to supply the pip configuration necessary to resolve all of the required Python package dependencies needed to fulfill the requirements for the training script (and later for inference).
# Ensure the source file 'pip.conf' in this project with the appropriate # index-url and credentials. RUN --mount=type=secret,id=pipconfig,target=/etc/pip.conf \ pip install -r /opt/ml/code/requirements.txt
Both the requirements.txt file and the pip.conf file are available in the git repository.
Update the pip.conf file as required with the credentials needed for the virtual PyPi repository in Artifactory, then build, tag and upload the custom Docker image to Artifactory. The following is an example command to accomplish this using the Dockerfile in the github repo. Remember to modify the Artifactory host name, the image name/tag, and the name of the virtual PyPi repository as needed for your environment.
docker build -t \ myartifactory.jfrog.io/sagemaker-docker-virtual/sagemaker/train:1.0_huggingface \ --secret id=pipconfig,src=pip.conf \ --build-arg "ARTIFACTORY_DOCKER_REGISTRY=myartifactory.jfrog.io" \ --build-arg "ARTIFACTORY_DOCKER_REPO=sagemaker-docker-virtual" . docker push myartifactory.jfrog.io/sagemaker-docker-virtual/sagemaker/train:1.0_huggingface
Note the format of the Docker tag. This format is required to identify the Artifactory registry and repository where the image will be pushed.