Similar to a SageMaker training job, a SageMaker-compatible Docker image is required when deploying a model for inference. The inference script is expected to be contained within the Docker image. In this example, the file entrypoint.py is also included that will start the model server. An example Dockerfile is available in the inference directory here.
Note that the inference Dockerfile uses the training image created in the previous section as a base image. This is certainly not required, but convenient for this example because it already contains the required dependencies needed for the inference script.
The following is an example command to build, tag, and upload the custom Docker image to Artifactory. Remember to modify the Artifactory host name, the image name/tag, and the name of the virtual PyPi repository as needed for your environment.
As with the training image, note the format of the Docker tag for the inference. This format is required to identify the Artifactory registry and repository where the image will be pushed.
Note that the inference Dockerfile uses the training image created in the previous section as a base image. This is certainly not required, but convenient for this example because it already contains the required dependencies needed for the inference script.
FROM ${ARTIFACTORY_DOCKER_REGISTRY}/${ARTIFACTORY_DOCKER_REPO}/sagemaker/train:1.0_huggingface
The following is an example command to build, tag, and upload the custom Docker image to Artifactory. Remember to modify the Artifactory host name, the image name/tag, and the name of the virtual PyPi repository as needed for your environment.
docker build -t \ myartifactory.jfrog.io/sagemaker-docker-virtual/sagemaker/inference-service:1.0_huggingface \ --build-arg "ARTIFACTORY_DOCKER_REGISTRY=myartifactory.jfrog.io" \ --build-arg "ARTIFACTORY_DOCKER_REPO=sagemaker-docker-virtual" . docker push myartifactory.jfrog.io/sagemaker-docker-virtual/sagemaker/inference-service:1.0_huggingface
As with the training image, note the format of the Docker tag for the inference. This format is required to identify the Artifactory registry and repository where the image will be pushed.