Skip to content

Talend TAC Orchestrator

Enterprise

Image$DATAOPS_TAC_RUNNER_IMAGE

The Talend TAC orchestrator runs existing Talend ETL jobs configured in the Talend Administration Center as part of a DataOps pipeline. This orchestrator allows the two technologies (Talend and DataOps.live) to co-exist.

Note

Before this orchestrator runs as part of a DataOps pipeline, you must register the ETL jobs in Talend's Job Conductor in your Talend Administration Center instance.

Usage

This YAML script demonstrates how to structure a typical TAC ETL pipeline job inside a DataOps pipeline.

pipelines/includes/local_includes/tac_jobs/my_tac_job.yml
"My TAC Job":
  extends:
    - .agent_tag
  stage: "My Stage"
  image: $DATAOPS_TAC_RUNNER_IMAGE
  variables:
    TAC_ACTION: START
    TAC_URL: XXXX
    TAC_USERNAME: XXXX
    TAC_PASSWORD: DATAOPS_VAULT(XXX)
    TAC_TASK_ID: XXXX
  script:
    - /dataops
  icon: ${TALEND_ICON}

Supported parameters

Parameter Required/Default Description
TAC_ACTION REQUIRED Must be START
TAC_URL REQUIRED The URL of the TAC instance to use
TAC_USERNAME REQUIRED The TAC user with access to start jobs in the Job Conductor
TAC_PASSWORD REQUIRED The TAC user's password
TAC_TASK_ID REQUIRED The job ID for the deployed task in the TAC Job Conductor
TAC_INSECURE Optional If set, it fetches the results of the deployed task on TAC in an insecure mode. It allows skipping the verification step to make sure the connection is secure before transferring the task results. For more information, see Curl Documentation.

Example jobs

This YAML script demonstrates a typical TAC use case, that is, starting an ETL pipeline to ingest data into a Snowflake database:

pipelines/includes/local_includes/tac_jobs/run_tac_job.yml
"Run TAC Job":
  extends:
    - .agent_tag
  stage: "Batch Ingestion"
  image: $DATAOPS_TAC_RUNNER_IMAGE
  variables:
    TAC_ACTION: START
    TAC_URL: https://tac1.example.com/org.talend.administration
    TAC_USERNAME: DATAOPS_VAULT(TALEND.TAC1.USERNAME)
    TAC_PASSWORD: DATAOPS_VAULT(TALEND.TAC1.PASSWORD)
    TAC_TASK_ID: 123456
  script:
    - /dataops
  icon: ${TALEND_ICON}