Skip to main content

Operational Hints and Tips

External Deletion of Object

If a SOLE-managed Snowflake object is deleted externally, it can lead to an error during the execution of pipelines. As external deletion does not update the state file of the resource, SOLE assumes that the object still exists. This leads to an error when SOLE tries to refresh the state of the object but is unable to find the object.

To handle such scenarios environment variable LIFECYCLE_STATE_RESET can be used to reset the state file for the Object Groups to which the object belongs.

Implicit Deletion of Database-Level Objects

A database managed by SOLE can be deleted in one of the following ways (via a Pipeline):

  1. Running Delete Jobs in Feature and Dev branches
  2. Deletion of Database by removing the definition from YAML files

If Database-Level Objects are not deleted by SOLE (by delete job), but implicitly (by Database deletion), it can lead to failure in a Pipeline.

To handle such scenarios the variable LIFECYCLE_STATE_RESET can be used to reset the state file for Database-Level Jobs.

Privilege Revokes on Objects shared in multiple Environments

If a managed resource is shared between environments (no environment-specific suffix added, see namespacing), then grants on the object would be reset in each pipeline as per the environment.

roles:
ROLE_1:
namespacing: suffix
comment: Role 1
warehouses:
WAREHOUSE_1:
comment: Warehouse 1
size: MEDIUM
namespacing: prefix
grants:
MONITOR:
- ROLE_1

In the above example, one role and one warehouse are managed by SOLE. The role ROLE_1 is created in each environment, whereas the warehouse WAREHOUSE_1 is created once and shared between environments.

If the pipeline runs in the PROD environment, then MONITOR on WAREHOUSE_1 would be granted to ROLE_1_PROD. When the same pipeline runs in the QA environment, the grant to ROLE_1_PROD is revoked and MONITOR is granted to ROLE_1_QA.

Resource Monitor Update

Due to a bug in Snowflake, a resource monitor cannot be created with a complete timestamp (YYYY-MM-DDTHH:MM:SS format) in parameters start_timestamp or end_timestamp. The parameters only support DATE (YYYY-MM-DD) at the time of the creation of a Resource Monitor.

As only DATE can be provided to create a Resource Monitor, the value of both start_timestamp and end_timestamp parameters must be a valid date (greater than the current date). After creation, the local state gets updated for the Resource Monitor. As Snowflake converts the date to a complete timestamp, the state has timestamp as the type for the parameters. Due to this, when the pipeline is run again, the resource monitor is updated given its configuration (DATE) differs from the value in the state (timestamp).

To remove the update, the value of the parameters start_timestamp and end_timestamp should be updated to the complete timestamp after the creation of the resource monitor.

Warning Message in Destroy-Plan and Destroy Jobs

After generation of destroy-plan or deletion of resources, terraform might log Warning messages such as below:

│ Warning: Resource targeting is in effect

│ You are creating a plan with the -target option, which means that the
│ result of this plan may not represent all of the changes requested by the
│ current configuration.

│ The -target option is not for routine use, and is provided only for
│ exceptional situations such as recovering from errors or mistakes, or when
│ Terraform specifically suggests to use it as part of an error message.
│ Warning: Applied changes may be incomplete

│ The plan was created with the -target option in effect, so some changes
│ requested in the configuration may have been ignored and the output values
│ may not be fully updated. Run the following command to verify that no other
│ changes are pending:
│ terraform plan

│ Note that the -target option is not suitable for routine use, and is
│ provided only for exceptional situations such as recovering from errors or
│ mistakes, or when Terraform specifically suggests to use it as part of an
│ error message.

These warning messages are generated when not all the objects in the Object Group are deleted.

If an Account-Level object's name does not contain a suffix (namespacing parameter set to value none or prefix), it would not be deleted when deleting job runs.

This is to ensure that objects shared by environments are not deleted by feature branches.