"Failed Quota"

When we get the description of the statefulset and replicaset, we see a similar event message that points to the source of the issue being related to a "failed quota".

ReplicaSet:

Events:
  Type     Reason        Age                  From                   Message
  ----     ------        ----                 ----                   -------
  Warning  FailedCreate  7m29s (x9 
 18m)  replicaset-controller  (combined from similar events): Error creating: pods "artifactory-ha-nginx-59dc69b597-6srtm" is forbidden: failed quota: resource-quotas: must specify limits.cpu for: nginx; limits.memory for: nginx; requests.cpu for: nginx; requests.memory for: nginx

PSQL StatefulSet:

Events:
  Type     Reason            Age                 From                    Message
  ----     ------            ----                ----                    -------
  Normal   SuccessfulCreate  22m                 statefulset-controller  create Claim data-artifactory-ha-postgresql-0 Pod artifactory-ha-postgresql-0 in StatefulSet artifactory-ha-postgresql success
  Warning  FailedCreate      62s (x19 over 22m)  statefulset-controller  create Pod artifactory-ha-postgresql-0 in StatefulSet artifactory-ha-postgresql failed error: pods "artifactory-ha-postgresql-0" is forbidden: failed quota: resource-quotas: must specify limits.cpu for: artifactory-ha-postgresql; limits.memory for: artifactory-ha-postgresql

Artifactory StatefulSet:

Events:
Type     Reason            Age                 From                    Message
  ----     ------            ----                ----                    -------
  Warning  FailedCreate      24m (x15 over 25m)  statefulset-controller  create Pod artifactory-ha-artifactory-ha-primary-0 in StatefulSet artifactory-ha-artifactory-ha-primary failed error: pods "artifactory-ha-artifactory-ha-primary-0" is forbidden: failed quota: resource-quotas: must specify limits.cpu for: artifactory-ha,event,frontend,jfconnect,metadata,migration-artifactory-ha,observability,router; limits.memory for: artifactory-ha,event,frontend,jfconnect,metadata,migration-artifactory-ha,observability,router; requests.cpu for: artifactory-ha,event,frontend,jfconnect,metadata,migration-artifactory-ha,observability,router; requests.memory for: artifactory-ha,event,frontend,jfconnect,metadata,migration-artifactory-ha,observability,router

This error can occur when we have defined resource quotas on the namespace but are missing limits for one or more resources.

Let's check the resourceQuotas in the namespace:

kubectl get resourcequotas -n resources

NAME              AGE   REQUEST                                                LIMIT
resource-quotas   42m   pods: 0/2, requests.cpu: 0/2, requests.memory: 0/4Gi   limits.cpu: 0/4, limits.memory: 0/6Gi

Sure enough, we can see that we have a resourceQuota named "resource-quotas" and the related resource requests and limits.

If we check the descriptions of the statefulsets and replicasets, we can verify what we are setting for each of these resources. We can see in each description that we are failing to set limits for the following services:

• Postgresql
• Nginx
• Artifactory-ha (artifactory)
• Event
• Frontend
• JFConnect
• Metadata
• Migration-artifactory-ha
• Observability
• Router

Let's go ahead and use one of the sizing.yaml files to set requests and limits for the Artifactory resources to attempt to overcome this issue. As we're using the default values.yaml file, we know these limits aren't being set. The sizing yaml file should take care of much of these but we can see that the following limits are missing from the file:

• Nginx
• Postgresql
• Migration-artifactory-ha

If you notice, there is a "migration-artifactory-ha" service that isn't included in either the default values.yaml or the sizing yaml files. What is this? If we check the "artifactory-primary-statefulset.yaml" file in the Helm Chart repository in GitHub, we will see that it is pulling resources from "Values.artifactory.primary.resources", the same resource settings for the "artifactory" (artifactory-ha) service. Since the sizing yaml files add the limits for "artifactory" we don't need to make any changes to meet the limit requirements for "migration-artifactory-ha".

Let's add the missing limits to the artifactory-small.yaml file (size as necessary):

postgresql:
  resources:
    limits:
      memory: "256Mi"
      cpu: "250m"
nginx:
  resources:
    limits:
      memory: "256Mi"
      cpu: "250m"

After saving the file, let's apply it to the cluster:

helm upgrade --install artifactory-ha jfrog/artifactory-ha -f artifactory-small.yaml -n <namespace>

Checking now, we can see that our PSQL pod is getting created! But we're still missing our Artifactory pods.

kubectl get pods -n resources
NAME                          READY   STATUS    RESTARTS   AGE
artifactory-ha-postgresql-0   1/1     Running   0          2m
artifactory-ha-nginx-77d44487c4-xhz6z   0/1     Running   0          2m