Skip to content

Informatica Cloud Taskflow Orchestrator

Enterprise

Image$DATAOPS_INFORMATICACLOUD_RUNNER_IMAGE

The Informatica Cloud orchestrator triggers the start of an Informatica taskflow and monitors its execution as part of a DataOps pipeline. Using it, can you leverage all existing Informatica cloud taskflows into a DataOps pipeline.

An Informaticta taskflow controls the execution sequence of a data transfer task, dynamic mapping task, mapping task, PowerCenter task, or synchronization task based on the output of the previous task. A taskflow defines the control flow logic within Informatica cloud. It covers various data integration and transformation needs, including ETL and ELT for Snowflake, for Informatica.

Usage

The Informatica Cloud orchestrator orchestrates taskflows via the Informatica Cloud REST API.

The orchestrator's workflow is as follows:

  1. Triggers the specified taskflow using your Informatica cloud organization URL.
  2. Polls the status and fetches progress for the taskflow.
  3. Propagates the Informatica Cloud taskflow execution status to the DataOps pipeline.
pipelines/includes/local_includes/informatica_jobs/my_informatica_job.yml
"My Informatica taskflow job":
  extends:
    - .agent_tag
  stage: "Data Transformation"
  image: $DATAOPS_INFORMATICACLOUD_RUNNER_IMAGE
  variables:
    INFORMATICA_TASKFLOW_URL: https://<your_org>.informaticacloud.com/active-bpel/rt/TaskflowID
    INFORMATICA_TASK_TYPE: TASKFLOW
    INFORMATICA_USERNAME: <username>
    INFORMATICA_PASSWORD: <password>
  script:
    - /dataops
  icon: ${INFORMATICA_ICON}


We recommend configuring the DataOps pipeline to continue to run, if the Informatica Cloud taskflwo job succeeds, ensuring that the pipeline run does not transform any out-of-date data.

Supported parameters

Parameter Required/Default Description
INFORMATICA_USERNAME REQUIRED Username to access the Informatica Cloud API
INFORMATICA_PASSWORD REQUIRED Password to access the Informatica Cloud API
INFORMATICA_TASK_TYPE REQUIRED Currently only supports TASKFLOW
INFORMATICA_TASKFLOW_URL REQUIRED The REST API URL for the taskflow. Normally starting with dm-em.informaticacloud.com or dm-us.informaticacloud.com.
INFORMATICA_TASKFLOW_ARGS Optional Arguments passed to Informatica Cloud taskflow
INFORMATICA_TIMEOUT Optional. Defaults to 3600 Informatica Cloud taskflow timeout in seconds. If you increased the DataOps job timeout, set this to an equivalent value, and vice-versa.

Example jobs

This example demonstrates what a typical pipeline job looks like:

pipelines/includes/local_includes/informatica_jobs/my_informatica_job.yml
"Informatica data ingestion":
  extends:
    - .agent_tag
  stage: Data Ingestion
  image: $DATAOPS_INFORMATICACLOUD_RUNNER_IMAGE
  variables:
    INFORMATICA_TASKFLOW_URL: https://<your_org>.informaticacloud.com/active-bpel/rt/TaskflowID
    INFORMATICA_TASK_TYPE: TASKFLOW
    INFORMATICA_USERNAME: DATAOPS_VAULT(INFORMATICA.USERNAME)
    INFORMATICA_PASSWORD: DATAOPS_VAULT(INFORMATICA.PASSWORD)
  script:
    - /dataops
  icon: ${INFORMATICA_ICON}
pipelines/includes/local_includes/informatica_jobs/my_informatica_job.yml
"Informatica data ingestion":
  extends:
    - .agent_tag
  stage: Data Ingestion
  image: $DATAOPS_INFORMATICACLOUD_RUNNER_IMAGE
  variables:
    INFORMATICA_TASKFLOW_URL: https://<your_org>.informaticacloud.com/active-bpel/rt/TaskflowID
    INFORMATICA_TASK_TYPE: TASKFLOW
    INFORMATICA_USERNAME: DATAOPS_VAULT(INFORMATICA.USERNAME)
    INFORMATICA_PASSWORD: DATAOPS_VAULT(INFORMATICA.PASSWORD)
    INFORMATICA_TASKFLOW_ARGS: |
      {
        "arg_1": "value"
      }
  script:
    - /dataops
  icon: ${INFORMATICA_ICON}