The DataOps API Orchestrator is a flexible orchestrator that enables pipelines to interact with RESTful API services across a wide range of applications. The included curl and jq applications enable querying remote services using scripts as well as parsing/displaying the JSON output. It also interacts with the DataOps vault to allow vault variables to be used in these scripts.
Example uses of this orchestrator include:
- Interacting with a third-party data ingestion tool to trigger ingestion at the beginning of a pipeline
- Triggering a downstream reporting system to refresh the data at the end of a pipeline.
For instance, you can interface with a job's external endpoints as follows:
"My API Job":
stage: "My Stage"
- curl https://some.api.service/api/v1/widgets
This example is of the DataOps API orchestrator interfacing with an ingestion job's endpoint:
"My API ingestion job":
stage: "Batch Ingestion"
- curl https://myendpoint.com/dosomething
This second example uses the API orchestrator to call a third-party API:
"Call external API":
stage: "Call API"
- curl $MY_API_URL > $MY_WIDGETS_FILE
- jq $MY_WIDGETS_FILE
Host Dependencies (and Resources)