A pipeline can, if necessary, control through Pipelines DSL which nodes it or any individual step will execute in.
When an administrator user sets up node pools in Pipelines, one of the node pools is designated as the default. This is the set of nodes that all pipeline steps will execute in when no explicit node pool is specified by the pipeline or step.
If you need to run your pipeline steps in a specific node pool, you can specify this in your pipeline's DSL. You can also configure any step to run on a node from a specific node pool.
You may wish to specify a node pool when:
Your pipeline or step is resource-intensive, so you need to execute in a specific node pool that uses more powerful machines
You need to run the pipeline or step on a specific OS (e.g., Ubuntu or CentOS)
You need to run a pipeline or step in a specific node pool for security (for example, behind a firewall in your own data center)
Note
Steps that are specified to run in the same affinityGroup
must always run in the same node pool.
Node Pools
Your Pipelines configuration must have at least one node pool, with at least one node, for pipelines to execute. These must be set up by an administrator user, with one node pool set as the default. A typical configuration will have been set up with multiple node pools to provide different choices for operating system and computing platforms.
For example, an administrator user may have set up these node pools and made them available to you for executing your pipelines:
A static node pool of Ubuntu 20.04 nodes running self-hosted, set as the default node pool
A dynamic node pool of Ubuntu 20.04 nodes running on Amazon Web Services
A static node pool of CentOS 7 nodes running self-hosted
You can view the node pools available to you from the Pipelines Node Pools tab:
Example Pipeline
In your pipeline's DSL you can designate the pipeline or any individual steps to run in a node pool by its name.
Pipeline node pool: You can specify a specific node pool for the pipeline in the
configuration
block of the pipeline, using thenodePool
tag. When specified here, all steps in the pipeline will execute in nodes from that node pool.Step node pool: You can specify a specific node pool for any step by declaring the
nodePool
tag in the step'sconfiguration
block. This will override any node pool designation for the pipeline.
The following example pipeline specifies that all steps of the pipeline will execute in nodes from the dynamic node pool ubuntu20aws
on Amazon Web Services. The second step, however, overrides this as it is configured to run on an self-hosted node from the static node pool ubuntu20
.
pipelines.yml
pipelines: - name: select_nodepool_example configuration: nodePool: ubuntu20aws # Run in the dynamic node pool on AWS steps: - name: nodePool_step1 type: Bash execution: onExecute: - echo "$step_name is running in $step_platform on $step_node_pool_name" - name: nodePool_step2 type: Bash configuration: nodePool: ubuntu20 # Run in the static node pool self-hosted inputSteps: - name: nodePool_step1 execution: onExecute: - echo "$step_name is running in $step_platform on $step_node_pool_name" - name: nodePool_step3 type: Bash configuration: inputSteps: - name: nodePool_step1 execution: onExecute: - echo "$step_name is running in $step_platform on $step_node_pool_name"
When the pipeline is run:
Steps
nodePool_step1
andnodePool_step3
will execute on nodes inubuntu20aws
Step
nodePool_step2
will execute on a node inubuntu20