Skip to main content

DataOps.live Orchestrators Pre-Release Versions

Every month, we roll out a new version of the DataOps.live orchestrators. The monthly updates include new features, bug fixes, and performance improvements. They are introduced in two stages: pre-release and official release versions. Initially, the updates are accessible under a temporary tag called 5-latest, which is later promoted to the stable tag called 5-stable.

Typically, the transition from a pre-release 5-latest version to the official release 5-stable version takes from 4 to 7 working days. However, this timeline may vary depending on the complexity of the release.

The monthly release notes serve as a comprehensive source of information, providing updates on all changes once they are pre-released and officially released.

Why use orchestrators pre-release versions?​

Benefits​

The pre-release 5-latest version allows you to:

  • Take advantage of the latest features and improvements of the orchestrators while still having the option to use the current stable version.
  • Verify that existing pipelines work as expected with the most recent DataOps.live changes before they become the new default.
  • Allow time to adjust to any backward incompatible changes in your code.

Best practices​

While the official orchestrator release 5-stable is recommended for your production environment and your day-to-day development and testing, consider using a feature environment to:

  • Test new functionalities of the orchestrators not yet available in the stable official version.
  • validate the backward compatibility of your existing pipelines with the latest changes.

To specify a version of the orchestrators in your DataOps.live project, set the DATAOPS_RUNNER_IMAGE_TAG variable. If not set explicitly, the 5-stable version will be used in every environment by default.

Run a pipeline with the pre-release version 5-latest​

To test the latest changes available in the 5-latest version, run a pipeline in a feature branch with the DATAOPS_RUNNER_IMAGE_TAG variable set to 5-latest.

Set this variable either in the YAML configuration inside your project, like so:

pipelines/includes/config/variables.yml
variables:
  DATAOPS_RUNNER_IMAGE_TAG: 5-latest

Or, set it at runtime while running a pipeline through the DataOps.live platform:

set-orchestrator-version !!shadow!!

Once you have set the variable, you can run your pipeline as usual.

If you detect any backward incompatible changes in 5-latest, make the appropriate adjustments referenced in the release notes in your project or contact the DataOps.live Support Team.

Switch back to the official release version 5-stable​

If you have set the DATAOPS_RUNNER_IMAGE_TAG variable to 5-latest inside the project rather than at runtime, make sure you are not merging this change in any of your regular development, testing and/or production branches. Otherwise, you are at risk of using the 5-latest version in production before it is considered sufficiently stable.

Additionally, any code adjustments you’ve made in your project to accommodate changes present in 5-latest should not be merged to any of your regular development, testing and/or production branches until the 5-latest version is promoted to 5-stable.

To switch back to the stable version, set the DATAOPS_RUNNER_IMAGE_TAG variable back to 5-stable:

pipelines/includes/config/variables.yml
variables:
  DATAOPS_RUNNER_IMAGE_TAG: 5-stable

Orchestrator versions across environments​

One of the critical features of the DataOps.live platform is its environment management engine, which lets you create and manage multiple environments for your projects. Each environment is a separate, isolated workspace where you can run pipelines using the DataOps.live orchestrators.

This lets you separate your development, testing, and production environments and manage them independently. Additionally, you can specify the version of the DataOps.live orchestrators that you prefer to use for each environment.

Last updated on