Talend TAC Orchestrator

Type	Pre-Set
Image	$DATAOPS_TAC_RUNNER_IMAGE

The Talend TAC orchestrator is a pre-set orchestrator that executes existing Talend ETL jobs configured in the Talend Administration Center as part of a DataOps pipeline. This orchestrator allows the two technologies (Talend and DataOps) to co-exist.

note

Before being run as part of a DataOps pipeline, you must register these ETL jobs in Talend's Job Conductor found in your Talend Administration Center instance.

Usage

This YAML script demonstrates how to structure a typical TAC ETL pipeline job run inside a DataOps pipeline.

pipelines/includes/local_includes/tac_jobs/my_tac_job.yml

"My TAC Job":
  extends:
    - .agent_tag
  stage: "My Stage"
  image: $DATAOPS_TAC_RUNNER_IMAGE
  variables:
    TAC_ACTION: START
    TAC_URL: XXXX
    TAC_USERNAME: XXXX
    TAC_PASSWORD: DATAOPS_VAULT(XXX)
    TAC_TASK_ID: XXXX
  script:
    - /dataops
  icon: ${TALEND_ICON}

Supported parameters

Parameter	Required/Default	Description
TAC_ACTION	REQUIRED	Must be START
TAC_URL	REQUIRED	The URL of the TAC instance to use
TAC_USERNAME	REQUIRED	The TAC user with access to start jobs in the Job Conductor
TAC_PASSWORD	REQUIRED	The TAC user's password
TAC_TASK_ID	REQUIRED	The job ID for the deployed task in the TAC Job Conductor

Example jobs

This YAML script demonstrates a typical TAC use case, that is, starting an ETL pipeline to ingest data into a Snowflake database:

pipelines/includes/local_includes/tac_jobs/run_tac_job.yml

"Run TAC Job":
  extends:
    - .agent_tag
  stage: "Batch Ingestion"
  image: $DATAOPS_TAC_RUNNER_IMAGE
  variables:
    TAC_ACTION: START
    TAC_URL: https://tac1.example.com/org.talend.administration
    TAC_USERNAME: DATAOPS_VAULT(TALEND.TAC1.USERNAME)
    TAC_PASSWORD: DATAOPS_VAULT(TALEND.TAC1.PASSWORD)
    TAC_TASK_ID: 123456
  script:
    - /dataops
  icon: ${TALEND_ICON}

Project resources

None

Host dependencies (and Resources)

None