Release Notes December 2023
Orchestrators
Feature Data loading from a select query in Pipe
SOLE (Snowflake Object Lifecycle Engine) now supports the select_statement parameter in copy_options of the Pipe object which lets you load data from a select query.
See Copy Options for more information about supported parameters and examples of usage.
Feature Delta generations in the VaultSpeed orchestrator
The VaultSpeed orchestrator now supports the deployment of both delta and full code generations.
For more information, see the VaultSpeed Orchestrator documentation.
Bug Fix Case-sensitive database prefix in SOLE
The DATAOPS_PREFIX variable is now case sensitive and SOLE throws an error when DATAOPS_PREFIX contains any character that is not supported. For more information about the supported characters see the Namespacing documentation.
This change is incorporated within a bundled feature flag, which is optional, offering flexibility to our users to adopt and test new features while minimizing risks. For more information about all Dataops.live bundles, see Bundled Feature Flags.
You can activate it by adding the value 2023_10 to the variable DATAOPS_ENABLE_BEHAVIOR_CHANGE_BUNDLE in your project settings.
PubPrev Snowpark (Python) orchestrator
The Snowpark (Python) orchestrator, which provides an intuitive API for querying and processing data in a data pipeline, is now in public preview. It has broadly finished development and will not change significantly when fully released to all customers.
For more information, see the Snowpark (Python) orchestartor documentation.
EoL Deprecated orchestrators
The following orchestrators are now archived and will be completely removed from our data product platform with the January stable release in 2024:
- Python2 Orchestrator
- AutoIngestion Orchestrator
Data product platform
PriPrev DataOps.live Assist in private preview
We're excited to introduce DataOps.live Assist — your DataOps copilot powered by artificial intelligence that transforms your user experience on the data product platform.
DataOps.live Assist is a tool for different team members, offering benefits including:
- Boost productivity by generating models from engineers' natural language inputs, describing models missing documentation, and explaining complex SQL.
- Speed up the review and approval of merge requests (MR) by summarizing and describing complex change requests.
- Analyze data pipeline failures, identify root causes, and provide natural language steps for incident resolution.
Be among the first to experience DataOps.live Assist. Join our private preview by emailing us at privatepreview@dataops.live, and your valuable feedback will shape its development.
Once you join, DataOps.live Assist is enabled by default for all users. If you have specific preferences and wish to opt out, email us at privatepreview@dataops.live.
See the DataOps.live Assist documentation for more information.
DataOps CLI (SOLE Generator)
Feature Enable overwriting the default database in SOLE Generator
You can now overwrite the default database for all objects in the SOLE Generator configuration file using the
--database argument. This value overwrites the database defined in the DDL query of the parsed objects.
See the Overwriting default database documentation for more information.
Bug Fix Bug fixes
- Fixed invalid quoting in statements of Procedures and Functions.
- Fixed parsing of Procedures and Functions having
CREATE OR REPLACE TABLEin their statement.