Release Notes September 2024
Improvement DataOps Runner for Snowpark Container Services now supports multiple runners
We are making a new version of the DataOps Runner available starting with the September release. In this release, we added support for setting up multiple runners per Snowflake account. A typical use case is environment segregation, and another is segregating runners per data product.
Learn how to now set up the DataOps Runner for Snowpark Container Services with multiple runners.
Orchestrators
The below Orchestrators release notes provide information about new features and updates in the pre-release version 5-latest.
These updates are set available as an opportunity to test and validate that things work as expected in your environment before we promote them to the official release 5-stable.
To test orchestrators with 5-latest, simply set DATAOPS_RUNNER_IMAGE_TAG to 5-latest in your project's variables.yml configuration file.
See Orchestrators Pre-Release Versions for more information.
Feature MATE with dbt version 1.7
We've rolled out dbt version 1.7 as the default for the MATE orchestrator and MATE packages, taking your DataOps experience to a whole new level.
If you want to continue using the previous dbt version, you can easily switch back to version 1.5 by following the instructions in the MATE orchestrator documentation.
Feature Support of dbt 1.8
We have introduced dbt version 1.8 support across our transformation orchestrator MATE and the MATE packages.
See the MATE Orchestrator and MATE Packages documentation for more information.
Feature Coalesce orchestrator
We've introduced a new Coalesce orchestrator to help you visually transform all your Snowflake data.
See the new Coalesce orchestrator documentation for for all details.
Feature Allow to declare views in SOLE for Data Products
Besides managing grants on views you now can also declare views with SOLE. We support direct definition in the statement parameter and support using the !dataops.include directive for long view definitions.
See the updated View documentation for more details and examples.
Improvement Key pair authentication in Snowflake orchestrator
The Snowflake orchestrator sees support for key-pair authenticating against Snowflake. Support for this additional authentication comes in useful for service accounts where do not want to use passwords.
See the Snowflake orchestrator documentation for more details.
Improvement New metadata fields in Collibra orchestrator
The metadata for the Collibra orchestrator has now been enriched with additional JSON fields.
Bug Fix Bug Fixes in SOLE and Orchestrators
- Disabled wrapping of the SnowSQL output in the VaultSpeed orchestrator.
- Fixed an issue with the
with_tagsproperty for newly created Tables in SOLE.