Why do we need node pools to be installed on a different host in the JFrog Pipelines environment?

Balaji Satish
2019-07-02 07:22


Node pools in JFrog pipelines is a group of nodes that helps us in running build steps simultaneously on different nodes.

It is a requirement that the node pools have to be hosted on a different host other than the Pipeline services host. 



Node pools are required to be installed on a different host other than the Pipeline service host because of the docker services. These docker services configured on the node pools are different than the services hosted on the Pipeline service host machine. The node pools services are required for the build jobs and the services on the Pipeline for the other internal Pipeline services such as WWW and API.

By installing them on the same host machine it may cause the required docker services to deleted and reinitialized. This is because on the node pools the docker services are cleaned up on every pipeline job run. 

If in case we do install both the Pipeline services and node pool on the same host machine, we would tend to use the docker daemon to manage both pipeline and node pool services, which is not a good practice. This would bring us to a scenario where we would be running user builds, example:- update or remove docker containers, which could remove the services required by the Pipeline service.