DataOps Bundled Feature Flags

Bundled feature flags in DataOps.live provides a structured and efficient approach to managing and deploying features that include behavioral changes that may break backward compatibility. When activated, bundled feature flags impact all the components of the data product platform associated with that particular bundle.

Why to use bundled feature flags?

Bundled feature flags offer control and flexibility, reducing complexity and providing users with a more efficient way to adopt and test new features while minimizing risks. Instead of toggling each feature individually, you can control multiple ones as a group, making activating or deactivating them easier by simply using the variable DATAOPS_ENABLE_BEHAVIOR_CHANGE_BUNDLE.

Feature flags facilitate efficient testing of sets of features. You can toggle on or off entire bundles during testing phases, streamlining the testing process. If any bundled features cause issues or conflicts, you can quickly revert the entire bundle, reducing the risk of unexpected problems affecting your operations.

See Project variables for information about where to set this variable and what value to use to activate a specific bundled feature flag.

DataOps.live bundles

Here is a list of the behavioral changes in all the released bundled feature flags.

Bundle 2024_05

Enforce strict validation for table_grants and schema_grants in SOLE

Previously, the Snowflake Object Lifecycle Engine (SOLE) did not prevent user misconfiguration in table_grants and schema_grants.

With the feature flag activated, you can now ensure that misconfiguration in table_grants and schema_grants is prevented. This is achieved by issuing a warning in the production branch pipeline and an error in the pipelines of other branches.

Bundle 2023_10

Case-sensitive database prefix in SOLE

Previously, the Snowflake Object Lifecycle Engine (SOLE) was adding the database prefix and suffix incorrectly when lowercase characters were present in the DATAOPS_PREFIX variable. For example, if DATAOPS_PREFIX was set to DataOps (rather than the more conventional DATAOPS), the database created by main pipelines would be named DATAOPS_DATAOPS_PROD_PROD.

Now, with the feature flag activated, SOLE makes the DATAOPS_PREFIX case sensitive and throws an error when DATAOPS_PREFIX contains any character that is not supported. The supported characters are uppercase letters, underscore and digits.

Bundle 2023_08

Empty YAML files

Previously, empty YAML files or empty lists in YAML files in your configuration caused compile errors.

With the feature flag activated, the Snowflake Object Lifecycle Engine (SOLE) handles empty lists and null values identically.

Revoking grants

Previously, SOLE did not revoke grants on the privileges of Snowflake objects when an empty list was provided.

With the feature flag activated, you can revoke all privileges with an empty grants: list.

SOLE in feature branch

Previously, when SOLE runs in a feature branch with DATAOPS_FEATURE_BRANCH_NONDB_ENV_NAME set to DEV, account-level resources are NOT reused from the DEV environment and are created for each feature branch. And when the feature branch teardown job runs, all account-level resources (apart from the main DB) are dropped from the DEV environment.

With the feature flag activated, SOLE doesn't create DEV resources in a feature branch, and, more importantly, it doesn't teardown DEV resources in a feature branch.