Utils Orchestrator

Professional

Enterprise

Image	$DATAOPS_UTILS_RUNNER_IMAGE

The Utils orchestrator contains various tools and command-line utilities, including:

• curl
• git
• gnupg
• jq
• lftp
• lsb-release
• openssh-client
• perl
• python3
• sshpass
• terraform
• unzip
• wget

Also included are the following Python libraries:

• jinja2
• pyyaml
• snowflake-connector-python

Usage

pipelines/includes/local_includes/utils_jobs/my_utils_job.yml

"My Utils Job":
  extends:
    - .should_run_ingestion
    - .agent_tag
  stage: "Batch Ingestion"
  image: $DATAOPS_UTILS_RUNNER_IMAGE
  variables:
  script:
    - echo "This is My Utils Job"
  icon: ${UTIL_ICON}

You can also create your own scripts in your repo, such as /scripts/myscript.sh and run this from a job. For example:

pipelines/includes/local_includes/utils_jobs/my_utils_job.yml

"My Utils Job":
  extends:
    - .should_run_ingestion
    - .agent_tag
  stage: "Batch Ingestion"
  image: $DATAOPS_UTILS_RUNNER_IMAGE
  variables:
  script:
    - . ./scripts/myscript.sh
  icon: ${UTIL_ICON}

Supported parameters

None