Skip to main content

Utils Orchestrator

TypeFlexible
Image$DATAOPS_UTILS_RUNNER_IMAGE

This flexible orchestrator contains various tools and command-line utilities, including:

  • ansible
  • curl
  • git
  • gnupg
  • jq
  • lftp
  • lsb-release
  • openssh-client
  • perl
  • python3
  • sshpass
  • terraform
  • unzip
  • wget

Also included are the following Python libraries:

  • jinja2
  • pyyaml
  • snowflake-connector-python

Usage

pipelines/includes/local_includes/utils_jobs/my_utils_job.yml
"My Utils Job":
extends:
- .should_run_ingestion
- .agent_tag
stage: "Batch Ingestion"
image: $DATAOPS_UTILS_RUNNER_IMAGE
variables:
script:
- echo "This is My Utils Job"
icon: ${UTIL_ICON}

You can also create your own scripts in your repo, such as /scripts/myscript.sh and run this from a job. For example:

pipelines/includes/local_includes/utils_jobs/my_utils_job.yml
"My Utils Job":
extends:
- .should_run_ingestion
- .agent_tag
stage: "Batch Ingestion"
image: $DATAOPS_UTILS_RUNNER_IMAGE
variables:
script:
- . ./scripts/myscript.sh
icon: ${UTIL_ICON}

Supported Parameters

None

Project Resources

None

Host Dependencies (and Resources)

None