Explicit Object Deletion
This mechanism helps you protect your data and avoid accidental deletions. It ensures that Snowflake objects aren't deleted unexpectedly. Common scenarios involving unintentional deletes include:
- Remove the code related to an object, database, or user
- Comment out the code related to an object, database, or user
- Resolve Git merge conflicts incorrectly
- Change the name of the object or Database
SOLE prevents the accidental deletion of your data by enforcing stricter rules on deleting objects in Snowflake. This topic discusses the following:
Enabling explicit deletion
To enable this feature, set the variable DATAOPS_PREVENT_OBJECT_DELETION
to 1
in your project and commit to the variables.yml
file. See Project Variables for more information.
Preventing accidental deletion of objects
When you enable this feature, you can delete one or more objects explicitly by setting the deleted
attribute on them.
This lets SOLE know that the deletion is intentional.
databases:
DATABASE_1:
comment: "new"
deleted: true
Deleting a parent object will also delete any child objects even if they are not marked as deleted.
In the following example, even though only the database is marked as deleted, the schema will also be deleted because it belongs to the database.
databases:
DATABASE_1:
comment: "new"
deleted: true
schemas:
SCHEMA_1:
# deleted: true
comment: schema1
grants:
CREATE TABLE:
- DATAOPS_SOLE_ADMIN_ROLE
In the same way, having this feature enabled lets SOLE know if the deletion is unintentional. If part of your configuration is removed, this will be considered a missing object that hasn't been explicitly marked to be deleted and will cause the SOLE job to fail and thus protects you against data loss.
For example, if you manually remove the configuration above for DATABASE_1
, the SOLE job will fail, and you should see some logging like the following:
[ DATABASE ] - [WARNING] - [16:25:59.0654] - Found missing object: DATABASE_1
[ DATABASE ] - [ ERROR] - [16:25:59.0654] - Found missing objects that would be unintentionally deleted. See above for details.
Moving objects between projects
Sometimes, you may want to stop managing an object in SOLE so that you can handle it manually or move it to a different project. To do this:
Set the object
manage_mode
property tonone
.dataops/snowflake/database.ymldatabases:
DATABASE_3:
comment: DATABASE 3
manage_mode: noneSOLE stops managing the object but it will still exist in Snowflake. Then if you were to remove this configuration from your project, SOLE would not complain as it's no longer managing the object and thus cannot be unintentionally deleted.
Add the object to SOLE in another project, remove
manage_mode
, and run the pipeline.The object is imported into the new project and you can start managing it there.