ARTIFACTORY: How to fix Terraform backend state file decryption failures after master.key rotation
OverviewAfter rotating the Artifactory master.key as part of routine security hardening or compliance procedures, operations against existing Terraform backend repositories, such as terraform plan, terraform apply, or retrieving state files, may fail due to state file decryption errors. This affects any Terraform Backend repository where encryptStates is enabled.
Symptoms & error messages
-
Terraform CLI fails with decryption or state retrieval errors when communicating with the Artifactory Terraform Backend. -
Artifactory logs report errors decrypting payload data for state files associated with Terraform backend repositories.
Cause
When the encryptStates setting is enabled on a Terraform Backend repository, Artifactory encrypts state files at rest using the cluster's active master.key.During a standard master.key rotation, the key itself is updated, but previously encrypted Terraform state files are not automatically re-encrypted with the new key in real-time. As a result, Artifactory attempts to decrypt the existing state files using the new master.key and fails.
Step-by-step solution
To safely execute a master.key rotation without compromising Terraform state file accessibility, follow the pre-rotation and post-rotation workflow below using the Update Repository Configuration REST API.
Phase 1: Pre-Rotation Preparation
-
Disable the encryptStates flag by setting it to false on the Terraform Backend repository, using the Update Repository Configuration API:
bash
curl -X POST "https://<ARTIFACTORY_URL>/artifactory/api/repositories/<REPO_KEY>" \
-H "Content-Type: application/json" \
-u <USERNAME>:<ACCESS_TOKEN> \
-d '{
"encryptStates": false
}'Note:
The default value for encryptStates is true.Note:
Disabling this setting decrypts the stored state files and temporarily stores them without master.key encryption. Ensure access controls and repository permissions are strictly maintained during this maintenance window.
Phase 2: Rotate the master.key
-
Perform the standard master.key rotation procedure across your Artifactory cluster nodes in accordance with standard operating procedures following this documentation. -
Verify that the Artifactory service has successfully initialized with the newly generated master.key.
Phase 3: Post-Rotation Re-encryption
-
Re-enable the encryptStates flag on the same repository by setting it back to true, using the same API:
bash
curl -X POST "https://<ARTIFACTORY_URL>/artifactory/api/repositories/<REPO_KEY>" \
-H "Content-Type: application/json" \
-u <USERNAME>:<ACCESS_TOKEN> \
-d '{
"encryptStates": true
}'
Verification
-
Confirm that Terraform client operations (terraform plan / terraform refresh) succeed against the repository. -
Confirm that state files are securely re-encrypted with the new master.key.