Managing dependencies

Sometimes you need more complex triggers than a simple pipeline of stages and jobs. In particular, you may want a pipeline to trigger based on the result of a stage in another pipeline. This is possible by adding pipelines as materials.

Creating a dependency

Say we have two pipelines – upstream_pipeline and downstream_pipeline . We want downstream_pipeline to automatically trigger following the successful completion of the stage AutoStage1 in pipeline upstream_pipeline. Here’s how we’d achieve this:

  • Navigate to the Admin section.
  • On the Pipelines screen, locate the downstream_pipeline pipeline and Edit it.
  • Click on the Materials tab.
  • Add a new pipeline dependency material by clicking Add Material and then selecting Pipeline .
  • You’ll be presented with an Add Material popup (see screenshot)
  • Enter upstream_pipeline [AutoStage1] in the Pipeline [stage] field (it can also auto-complete)

pipeline_add_material
Power users can also configure this via the Source XML tab on the Admin section:

<pipeline name="downstream_pipeline">  
  <materials>  
    <pipeline pipelineName="upstream_pipeline" stageName="AutoStage1"/>  
  </materials>
  ...
</pipeline>

Now, when the stage “AutoStage1” of “upstream_pipeline” completes, the pipeline “downstream_pipeline” will start building. The Pipeline Dependency visualization shows you all the downstream instances that were triggered off the upstream instance (label 14) currently being viewed.

dependent_build
If you want to view the materials that are associated with “downstream_pipeline”, the pipeline details page for that specific instance of the downstream pipeline will show you all this information.

downstream_pipeline

Fetching artifacts from an upstream pipeline

Go can automatically fetch artifacts from a previous stage of the current pipeline or from another specified pipeline. This is useful when a pipeline depends on binaries that are produced earlier in the pipeline.

Note that you can not specify two (or more) dependencies for the same upstream pipeline.

For example, in the following configuration, when the stage “AutoStage1” of pipeline “upstream_pipeline” passes, the pipeline “downstream_pipeline” starts, and the artifacts are fetched from the upstream pipeline in the stage ‘Stage’ of “downstream_pipeline”. You can see the exact pipeline and stage that triggered this in the sub-tab ‘Materials’.

You can do this via the admin screens for the respective pipelines. You’ll need to first define the artifact in the “upstream_pipeline” at the job level:

job_artifacts
Then, you’ll want to retrieve (fetch) that artifact from within the “downstream_pipeline.” You can do this by creating a “Fetch Artifact” task within a job in that pipeline. Since you have already defined “upstream_pipeline” as a dependency material, artifacts from that pipeline are accessible in this pipeline.

task_fetch_artifact
For power users, here’s how you can configure this via Source XML:

<pipeline name="upstream_pipeline">  
  <materials>  
    <svn url="...."/>  
  </materials>
  ...
  <stage name="AutoStage1">  
    <job name="firstJob">  
      <tasks>  
        <nant />  
      </tasks>
      <artifacts>  
        <artifact src="target/commonlib.dll" dest="pkg"/>  
      </artifacts>
    </job>
  </stage>
</pipeline>
<pipeline name="downstream_pipeline">  
  <materials>  
    <pipeline pipelineName="upstream_pipeline" stageName="AutoStage1"/>  
  </materials>
  <stage name="Stage">  
    <job name="firstJob">  
      <tasks>  
        <fetchartifact pipeline="upstream_pipeline" stage="AutoStage1" job="firstJob" srcfile="pkg/commonlib.dll" dest="libs"/>  
      </tasks>
    </job>
  </stage>
  ...
</pipeline>