Skip to main content

How to Auto-Cancel Duplicate Pipelines

If an instance of a pipeline starts before the previous instance has been completed, use the outlined approach to abort the new instance so that multiple pipelines do not run concurrently.

Note

It is, of course, possible to use resource groups to "pause" the new pipeline and wait for the previous jobs to complete. But this approach can be safer if we run hourly-ci.yml on a schedule and each run takes longer than 60 minutes to complete.

Sample Code​

The following job can be included in a pipeline to check the currently-running pipelines and abort if there's already one running the same pipeline file on the same branch. The newer pipeline will fail with an error so that notifications will be sent (depending on the project's configuration), and action can be taken as appropriate.

pipelines/includes/local_includes/auto_abort.yml
Check for Running Pipeline:
  extends: .agent_tag
  image: $DATAOPS_UTILS_RUNNER_IMAGE
  stage: Pipeline Initialisation
  variables:
    API_AUTH_HEADER: "PRIVATE-TOKEN: $LOCAL_PROJECT_TOKEN"
    API_URL_PROJECT: https://app.dataops.live/api/v4/projects/$CI_PROJECT_ID
    PIPELINE_URL_BASE: https://app.dataops.live/dataops-internal/sam/am-i-running/-/pipelines/
  script: |
    echo "This is project $CI_PROJECT_ID (ref $CI_COMMIT_REF_NAME) running pipeline $CI_PIPELINE_ID ($_PIPELINE_FILE_NAME)"
    api_response=$(curl -s -H "$API_AUTH_HEADER" "$API_URL_PROJECT/pipelines?ref=$CI_COMMIT_REF_NAME&status=running")
    pipeline_ids=$(echo "$api_response" | jq -r --arg current_id "$CI_PIPELINE_ID" --arg pname "$_PIPELINE_FILE_NAME" 'if type == "array" then .[] | select(.id != ($current_id|tonumber)) | select(.name == $pname) | .id else empty end')
    for id in $pipeline_ids; do
      echo -e "\e[31m[ERROR] Another pipeline is also running $_PIPELINE_FILE_NAME in $CI_COMMIT_REF_NAME:\e[0m $PIPELINE_URL_BASE$id"
      exit 1
    done
    echo -e "\e[92m[OK] No other pipelines are running $_PIPELINE_FILE_NAME in $CI_COMMIT_REF_NAME\e[0m"

API Authentication​

Since the scope of the built-in CI_JOB_TOKEN variable does not cover these parts of the DataOps.live API, it is necessary to provide alternative credentials. One approach that we have used here is to create a project access token against this project, using the api scope, and store this in a project variable (we have used LOCAL_PROJECT_TOKEN in this code).

Explanation of the Code​

The following variables have been defined in the job for clarity:

  • API_AUTH_HEADER Header passed to the curl command for authentication against the DataOps.live API.
  • API_URL_PROJECT Base URL fragment for the DataOps.live API endpoint that relates to the current project.
  • PIPELINE_URL_BASE Base URL fragment for constructing data product platform URLs to pipelines (for convenience in error messages).
echo "This is project $CI_PROJECT_ID (ref $CI_COMMIT_REF_NAME) running pipeline $CI_PIPELINE_ID ($_PIPELINE_FILE_NAME)"
api_response=$(curl -s -H "$API_AUTH_HEADER" "$API_URL_PROJECT/pipelines?ref=$CI_COMMIT_REF_NAME&status=running")
pipeline_ids=$(echo "$api_response" | jq -r --arg current_id "$CI_PIPELINE_ID" --arg pname "$_PIPELINE_FILE_NAME" 'if type == "array" then .[] | select(.id != ($current_id|tonumber)) | select(.name == $pname) | .id else empty end')
for id in $pipeline_ids; do
  echo -e "\e[31m[ERROR] Another pipeline is also running $_PIPELINE_FILE_NAME in $CI_COMMIT_REF_NAME:\e[0m $PIPELINE_URL_BASE$id"
  exit 1
done
echo -e "\e[92m[OK] No other pipelines are running $_PIPELINE_FILE_NAME in $CI_COMMIT_REF_NAME\e[0m"
  • Line 2: Calls the pipelines API and stores the raw JSON response in api_response.
  • Line 3: Pipes api_response through jq with two shell variables injected — $current_id (the current pipeline ID) and $pname (the current pipeline file name). If the response is an array, it filters out the current pipeline by ID and then filters to only pipelines whose .name matches $_PIPELINE_FILE_NAME, returning their IDs. If the response is not an array (e.g. an API error object), it returns nothing safely via empty.
  • Lines 4-7: For each matching pipeline ID found, print an error message with a direct link to that pipeline and exit with a non-zero code, causing the job and pipeline to fail.
Last updated on