Release Notes October 2024
Orchestrators
The below Orchestrators release notes provide information about new features and updates in the pre-release version 5-latest.
These updates are set available as an opportunity to test and validate that things work as expected in your environment before we promote them to the official release 5-stable.
To test orchestrators with 5-latest, simply set DATAOPS_RUNNER_IMAGE_TAG to 5-latest in your project's variables.yml configuration file.
See Orchestrators Pre-Release Versions for more information.
Improvement Orchestrators Transitioning to Ubuntu 24.04 LTS
We have upgraded the base of all orchestrators to Ubuntu 24.04 LTS, which will provide the most secure, reliable, and feature-rich environment for your data operations.
Check our community announcement for more details.
Feature Streams on Views in SOLE
We have introduced a new on_view parameter for Stream objects in
SOLE letting you define streams on views.
See the Stream documentation for more information.
Improvement Key pair authentication in data.world catalog orchestrator
The data.world catalog orchestrator now supports key-pair authentication against Snowflake. Support for this additional authentication is required for service accounts where you cannot use passwords.
See the data.world catalog orchestrator documentation for more details.
Bug Fix New dataops.maximum_rows_v2 test in MATE
We have introduced a new dataops.maximum_rows_v2 version of the existing dataops.maximum_rows test in the
MATE orchestrator. The new test fixes the invalid behavior of
dataops.maximum_rows, but both versions will temporarily be available for backward compatibility. Migrate to
dataops.maximum_rows_v2 at your earliest convenience.
See the MATE automated data testing documentation for more information.
Bug Fix Fixed the after parameter of Task objects in SOLE
The behaviour of the after parameter of Tasks has been updated and
you can now use it to define predecessor tasks via SOLE.
EoL ansible no longer available by default
The ansible package is no longer available by default in the Utils,
Azure, and AWS orchestrators. If you
want to keep using it, you can install it in in your DataOps.live project using the apt install ansible command.
DataOps CLI (SOLE Generator)
Feature Enable storing statements of Procedures and Functions in external files
The SOLE Generator now supports a new input argument --inline-statement-limit
(-is) that allows you to store the statements of procedures and
functions in separate files. Each statement longer than the specified
limit will be stored in a separate file. This feature is useful when you want to keep your SOLE files clean and readable.
For more information, see the Opt-In Feature for Managing Procedure and Function Statements article in the DataOps.live Community.
You can also refer to the SOLE Generator documentation for some usage examples.
Bug Fix Bug fixes
- Fixed invalid parsing of procedures having default argument values.
- Fixed invalid parsing of objects passed via the
--input-dir(-d) argument insole-dp gen. - Fixed missing arguments while parsing procedures.
Operate
Improvement General improvements and security patching
- Improved internal system monitoring.
- Security patching and standard system package updates.