A pipeline's run state is persistent only between steps within the same run. Information stored in one step is available to subsequent steps in the run of that pipeline. After the run is complete, state may be downloaded when viewing the steps in that run but will not be available in later runs or other pipelines.
To preserve state across steps, use the utility functions for run state management.
Pipelines supports two types of run state information that can be preserved between steps.
add_run_variables utility function, you can store a key-value pair to the run state. That key-value pair will automatically be available to all subsequent steps in the run as an environment variable. It will not be available to steps that do not have the step that added the variable as an input, either directly or through other steps or resources.
add_run_files utility function, a step can store a file in the run state. Any subsequent step can then use the
restore_run_files function to retrieve the file from the run state. Files are available to steps in the same run whether or not the step that added the files is an input to the later step. Run state may be downloaded for an individual step, consisting of the files either uploaded or downloaded by that step.