Stitch Orchestrator

Enterprise

Image	$DATAOPS_STITCH_RUNNER_IMAGE

The Stitch orchestrator integrates and runs Stitch ETL pipelines as part of a DataOps pipeline.

note

Stitch is an ingestion technology that is now part of Talend.

Usage

This YAML script demonstrates how to structure a typical Stitch ETL pipeline job run inside a DataOps pipeline.

pipelines/includes/local_includes/stitch-jobs/my_stitch_job.yml

"My Stitch Job":
  extends:
    - .agent_tag
  stage: "My Stage"
  image: $DATAOPS_STITCH_RUNNER_IMAGE
  variables:
    STITCH_ACTION: START
    STITCH_SOURCE_ID: XXXX
    STITCH_ACCESS_TOKEN: DATAOPS_VAULT(XXXX)
  script:
    - /dataops
  icon: ${STITCH_ICON}

Supported parameters

Parameter	Required/Default	Description
STITCH_ACTION	REQUIRED	Must be START
STITCH_SOURCE_ID	REQUIRED	Numeric identifier of the specific Stitch integration to trigger. Grab this from the URL when editing the integration settings in Stitch. See app.stitchdata.com for more details
STITCH_ACCESS_TOKEN	REQUIRED	Stitch API access token (see the Stitch docs)

Example jobs

This YAML script demonstrates a typical Stitch use case, that is, starting an ETL pipeline to ingest data into a Snowflake database:

pipelines/includes/local_includes/stitch_jobs/stitch_ingestion.yml

"Start Stitch Ingestion":
  extends:
    - .agent_tag
  stage: "Batch Ingestion"
  image: $DATAOPS_STITCH_RUNNER_IMAGE
  variables:
    STITCH_ACTION: START
    STITCH_SOURCE_ID: 123456
    STITCH_ACCESS_TOKEN: DATAOPS_VAULT(STITCH.ACCESS_TOKEN)
  script:
    - /dataops
  icon: ${STITCH_ICON}