How JFrog Delivers Self-Service Cloud Environments for our Developers

The internal DevOps team at JFrog needed to provision cloud resources, create environments, and manage infrastructure for our developers.  Unfortunately, it involved wasting a significant amount of time on repetitive tasks, that was slowing down the pace of innovation and taking away our developers’ focus from building new features and industry leading products.

Here is a behind the scenes look at how the JFrog DevOps team addressed this challenge using Crossplane, an open-source project that enables management of infrastructure as code (IaC) with Kubernetes-native resources. By leveraging Crossplane, we were able to build an internal self-service platform that allows developers to easily create resources or environments without needing to worry about underlying infrastructure complexities.

What is Crossplane?

Before diving into the technical details, let’s first understand Crossplane. At its core, Crossplane is an open-source project that enables you to manage cloud resources and services using Kubernetes-native API objects. It’s an abstraction layer that enables developers to define infrastructure resources in the same way they manage applications – using the familiar Kubernetes interface and syntax.

Crossplane provides a declarative approach for provisioning infrastructure, which makes it easy to create and manage cloud resources such as databases, storage, networking, and even whole environments, directly from Kubernetes.

The Challenge of Provisioning Developer Resources

As part of the infrastructure team, we noticed that developers in our organization often had to request resources manually or wait for approval to provision new environments. The process was cumbersome, slow, and error-prone. Developers didn’t have the autonomy they needed, and teams were spending too much time on administrative tasks rather than focusing on coding to meet release dates.

Our goal was to create a self-service platform where developers could easily provision the resources or environments they needed without relying on DevOps or other infrastructure related  teams. The goal was to create a smooth, efficient experience that reduced the obstacles in creating cloud environments and increased developer productivity.

Crossplane Integration

Following our comprehensive research, we determined that Crossplane was the perfect tool for the job. By integrating this Go-based package into our Kubernetes environment, we could finally provide developers with a self-service experience while keeping full control over the infrastructure. Here’s how we leveraged Crossplane to meet our challenges:

1. Defining IaC with Crossplane’s Resource Management

The first step was defining the resources that developers would need. With Crossplane, we were able to write Kubernetes custom resources (CRs) that defined the infrastructure requirements. These resources provide an abstraction layer that significantly reduces complexity by enabling unified commands regardless of the cloud provider, allowing developers to request resources by running simple jobs through the creation of a YAML file. .

2. Creating a Self-Service Interface

Once the basic API and infrastructure resources were defined, we created a simple Jenkins job to apply Crossplane resource definitions.

Then we built a user-friendly job for developers that allowed them to input their resource specifications, such as  defining a database size, selecting cloud providers and submitting requests for new environments or resources.

Using the CrossplaneAPI to fill in the parameters of the Crossplane’s Composition, we were able to create reusable templates for the most common environment setups, making it easy for developers to provision new environments on their own by simply selecting from the predefined options.

For example, a developer could quickly create a new environment with JFrog Artifactory and JFrog Xray set up on top of a Kubernetes cluster, using a PostgreSQL database and object storage by selecting a pre-configured template.

3. Automating Provisioning and Management

Behind the scenes, we implemented a workflow to interact with the CrossplaneAPI results that  generated a Custom Resource (CR), stored in Git and using ArgoCD to apply  the CR to Crossplane, Once a request was submitted, Crossplane automatically took care of provisioning the necessary cloud resources and configuring them to the correct specifications by pulling images and helm charts stored in Artifactory. Since Crossplane abstracts cloud APIs, the entire process could happen automatically, reducing the need for human intervention.

Once the request is submitted, Crossplane handles the provisioning of the requested resources through a series of automated steps:

Steps involved in a typical Crossplane automated provisioning process

To better understand how this automation works, let’s breakdown what is happening behind the scenes:

a. Create Kubernetes Custom Resources (CRs)

In the background, Crossplane automatically converts the user request into Kubernetes Custom Resources (CRs). For example, if the developer requested a PostgreSQL database, Crossplane would generate a PostgreSQLInstance CR that defines the database configuration. These CRs are stored in a Kubernetes cluster, which Crossplane uses to manage the resources.

b. Crossplane Compositions

Crossplane uses Compositions to define higher-level abstractions of cloud resources. A Composition allows us to define reusable blueprints for infrastructure. For example, one composition might define how a fully managed PostgreSQL instance, an S3 bucket, and a Kubernetes cluster should be created together.

Code used for creation of a PostgreSQL instance, S3 bucket, and Kubernetes cluster

c. Reconciliation Loop and Resource Creation Operator

Once the CRs are created, Crossplane enters into a reconciliation loop, which is part of Kubernetes’ controller pattern. Crossplane continuously checks the state of the resources to ensure they are provisioned correctly. If something is missing or misconfigured, Crossplane will attempt to correct it by interacting with the cloud provider’s API and adjusting the resources accordingly.

This is a key benefit of using Crossplane: it maintains the desired state of resources and ensures consistency even when provisioning processes take time or face delays.

We also integrated it with GitOps workflows and DevOps processes with Artifactory, where any changes to the resources or environments were stored in version-controlled Git repositories and Artifactory to store images and helm charts. This ensured that developers could track changes, roll back configurations, and collaborate on infrastructure in the same way they would with application code.

4. Role-Based Access Control (RBAC) and Permissions

One of the key requirements for our self-service platform was ensuring that developers had appropriate levels of access based on their roles. Crossplane’s RBAC support allowed us to define fine-grained permissions for different users. And a secure access proxy ensured that developers could only provision and access to the resources they were created, while also allowing them to self-manage the resources they owned.

For example, developers working on different projects could only create and manage resources they created for their project, ensuring a secure and scalable environment.

5. Monitoring and Cost Management

To make sure that resources were being used efficiently, we integrated monitoring and cost management tools into the platform, and by tagging the “expiry” and “owner” we provided visibility into their usage and costs. This helped us identify any unused or underutilized resources, giving us the ability to optimize costs across the platform.

On top of that all infrastructure is created using predefined size classes (small/medium/large) that align with cloud cost best practices. No more ad-hoc requests with oversized resources. This alone reduced ~25% in cloud waste.

Diagram showing self service provisioning in a mixed on-prem and multi-cloud environment (Click to enlarge)

6. Time Saving – from Days to Minutes

Environment Type	Previous Manual Time	Self-Service Time
Dev/Test K8S Env	1–2 days	~10 minutes
PostgreSQL Database	Several hours	~5 minutes
Full Microservice Stack	1-2 days	~10 – 15 minutes

 

This shift resulted in:

• 85–90% reduction in provisioning time
• Developers self-serving infra with zero tickets
• DevOps focusing on platform improvements, not provisioning

10x Scalability and Additional Key Benefits

Incredibly, with the Artifactory and Crossplane integration in place, our team succeeded in creating an environment that could be scaled to 10 times its current capacity in a matter of minutes.

Developer platform at scale – readiness for 10x Growth

With our new architecture, we are now positioned to scale infrastructure while increasing operational efficiency by:

• Leveraging composable, reusable Crossplane compositions to handle 70% of resources provisioning requests related to creation and updates of development environments.
• Managing Infrastructure changes  via GitOps and Artifactory, bringing version control, auditability, and rollback capabilities across the entire software supply chain.

This significant improvement in development efficiency has already been proven in the wild  through automated responses to hundreds of environmental requests already, and we’re just getting started.

In addition to scalability, by using Crossplane to build our self-service platform, we realized several additional key benefits:

• Empowered Developers: Developers no longer have to wait for approval or depend on other teams to create environments. They now have full control over provisioning resources, reducing wait times and boosting productivity.
• Consistency and Standardization: Since resources are defined in code, we could ensure that the same standards and best practices were applied across the organization. The templates provided a way to enforce consistency across environments and contributed to a 25% reduction in cloud waste by eliminating over-provisioned and idle resources.
• Increased Efficiency: Automation reduced the manual effort required to provision resources, freeing up time for both developers and infrastructure teams to focus on more important tasks.
• Improved Cost Management: With visibility into resource usage and tagging to identify resource owners, we are now in a better position to manage costs and identify areas for optimization.

Conclusion

By leveraging Crossplane, we were able to create an internal self-service platform that allowed developers to quickly provision resources and create environments with minimal overhead and cost wise. This not only streamlined our development process but also empowered developers to be more independent and productive.

If you want to learn more about how to create self-service provisioning for your development environments using JFrog Artifactory then feel free to set up an integration demo or contact JFrog Professional Services at your convenience.