Python2 Orchestrator
Type | Flexible |
---|---|
Image | $DATAOPS_PYTHON2_RUNNER_IMAGE |
Feature Status | EoL |
The Python2 orchestrator is a flexible orchestrator and includes Python2 as well as the following libraries:
- requests
- responses
- NumPy
- pandas
Usage
This orchestrator is used to create scripts that run jobs. Therefore, you can create your Python script inside your DataOps project, such as python2 my-script.py
, and run it from a job. In order to pass variables into the job, set them in the YAML config file's variables
block. For example:
pipelines/includes/local_includes/python2_jobs/my_python2_job.yml
"My Python2 Job":
extends:
- .agent_tag
stage: "My Stage"
image: $DATAOPS_PYTHON2_RUNNER_IMAGE
variables:
SCRIPT_PATH: $CI_PROJECT_DIR/scripts/my-script.py
script: python2 my-script.py
icon: ${PYTHON_ICON}
Supported parameters
Parameter | Required or Optional (and default) | Description |
---|---|---|
DATAOPS_RUN_PYTHON_SCRIPT | Optional | Project path to a Python2 script |
FILE | Optional. Defaults to requirements.txt | Name of the file used to run pip install |
SPECIFIED_PATH | Optional. Defaults to $CI_PROJECT_DIR | Path to requirements.txt file |
Example jobs
The following example shows how to execute a Python2 script:
pipelines/includes/local_includes/python2_jobs/execute_python2_script.yml
"Execute Python2 Script":
extends:
- .agent_tag
stage: "Additional Configuration"
image: $DATAOPS_PYTHON2_RUNNER_IMAGE
variables:
SCRIPT_PATH: $CI_PROJECT_DIR/scripts/my-script.py
script: python2 $SCRIPT_PATH
icon: ${PYTHON_ICON}
To retrieve the secrets PATH.TO.VAR1
and PATH.TO.VAR2
from the DataOps Vault, add them as variables to the YAML config file as follows:
pipelines/includes/local_includes/python2_jobs/extract_from_vault.yml
"Extract From Vault":
extends:
- .agent_tag
image: $DATAOPS_PYTHON2_RUNNER_IMAGE
stage: "Additional Configuration"
variables:
DATAOPS_RUN_PYTHON_SCRIPT: $CI_PROJECT_DIR/scripts/my-script.py
VAR1: DATAOPS_VAULT(PATH.TO.VAR1)
VAR2: DATAOPS_VAULT(PATH.TO.VAR2)
script:
- /dataops
icon: ${PYTHON_ICON}
Project resources
None
Host Dependencies (and Resources)
None