Skip to content

Orchestrators Release Notes (September 22, 2022)

v5.7.0 (5-latest)

Feature SOLE

  • You can now easily apply tags on the Snowflake objects Database, Schema, and Table by using the new keyword with_tags. For example:
databases:
  DATABASE_1:
    schemas:
      SCHEMA_1:
        tables:
          TABLE_1:
            with_tags:
              DEPARTMENT: Sales
            columns:
              COLUMN_1:
                type: TEXT
                with_tags:
                  DEPARTMENT: Sales

Based on these configurations, SOLE generates and executes a set of statements to update or apply the tags to the related object. Refer to the SOLE tag management for a complete discussion of how to use the new capabilities.

  • Better validation techniques for the SOLE configuration are implemented:
  • Schema validation fails if an invalid or unsupported argument is specified
  • SOLE configuration key validation throws an error if the object identifier is not a string
  • Stage validation is added for the encryption parameter of an internal stage

Feature Orchestrators

  • The SecretsManger orchestrator now supports integration with Hashicorp Vault
  • A new Snowpark Python orchestrator has been introduced to help you query and process data during pipeline execution in private preview.

Improvement In SOLE and Orchestrators

  • Improved the usability of SOLE logging when it runs the first time and discovers all existing objects. Instead of logging many errors for objects it didn't know about yet, it now logs only warnings and limits the number of warnings to better indicate that this is expected behavior.
  • Updated logging to use the DataOps Logger in the Reporting orchestrator and improve the log info and layout
  • Supported attaching Network Policy to User and Account
  • TMC orchestrator:
  • Added seamless retries for the HTTP status codes 502, 503, and 504
  • Updated logging to warn you when expected errors happen, e.g., network issue

Bug Fix In MATE, SOLE, and Orchestrators