SOLE Lifecycle Actions
SOLE lifecycle actions manage the lifecycle of all Snowflake objects, from create to destroy. In other words, these actions inform the SOLE Orchestrator what behaviors to execute. This content describes the following possible options for the LIFECYCLE_ACTION parameter:
Once we have completed discussing these lifecycle actions, the last part of this topic is to look at how to prevent deleting the production database and the Snowflake user used by SOLE.
AGGREGATE
As an alternative to setting up individual jobs for each object group and lifecycle action, you can configure an aggregate action by setting the LIFECYCLE_ACTION to AGGREGATE. Doing this performs the following actions for all object hierarchies sequentially and in a single job:
This reduces the number of pipeline jobs to one job that manages the Snowflake object lifecycle.
As AGGREGATE acts on all the object hierarchies, you don't have to specify the parameter LIFECYCLE_MANAGE_OBJECT. However, other parameters, such as CONFIGURATION_DIR are still required.
Suppose you need a different process and want to review and approve any modifications planned for a Snowflake database before applying them. In that case, you can use the PLAN-ALL and APPLY-ALL actions. Check out Two-step plan and apply for more information.
COMPILE
When the LIFECYCLE_ACTION is set to COMPILE, SOLE generates the configuration required to initialize and manage Snowflake objects from the user-defined configurations. In other words, the COMPILE action tells SOLE to read all the YAML files defined by users and generate a machine-readable configuration that SOLE can execute to enact the user-defined Snowflake configuration.
SOLE uses these configurations to perform validation and dependency resolution, generate resource configurations and syntax in the relevant stages, and resolve the grants and revokes on Snowflake Objects. See Grants for more information on Snowflake object access privileges.
The following parameters are used to control the execution of the COMPILE action:
| Parameter | Required/Default | Set in | Description |
|---|---|---|---|
CONFIGURATION_DIR | REQUIRED | SOLE Orchestrator | The directory path where the Snowflake configuration files are located |
DATAOPS_PREFIX | REQUIRED | project settings | The prefix to be added to the account and database-level object hierarchies (except the default database) |
DATAOPS_ENV_NAME | REQUIRED | project settings | The suffix to be added to the account and database-level object hierarchies (except the default database) |
DATAOPS_SOLE_DEBUG | Optional | pipeline parameters | This parameter enables the debugging of SOLE without writing sensitive information to the logs. |
DATAOPS_DEBUG | Optional | pipeline parameters | If set, the DataOps debug logs are enabled |
VALIDATE
When the LIFECYCLE_ACTION is set to VALIDATE, SOLE validates the resource configurations generated during the COMPILE stage.
PLAN
SOLE plans the execution steps to create or update the managed Snowflake objects when the LIFECYCLE_ACTION is set to PLAN.
Before generating a plan, SOLE tries to import all the specified Snowflake objects if they exist. This import helps initialize or update the local state for existing Snowflake objects used to generate the plan. An import summary is displayed during and after the completion of the import.
Due to aliases and multiple format support, a plan might include data-type alias changes (in-place changes) for Snowflake Objects.
Also, suppose the state file is corrupted or it contains a discrepancy, such as an object being managed in the state file but deleted from Snowflake. In this case, the state file can be removed and re-initialized from the current Snowflake object configuration. To do so, set the LIFECYCLE_STATE_RESET parameter to any value for the pipeline to re-initialize the local state for all object hierarchy groups. This parameter can also be initialized for any individual PLAN job to reset the state for a particular object or object group.
The generated plan is provided to the APPLY action.
APPLY
When the LIFECYCLE_ACTION is set to APPLY, SOLE executes the steps specified by the PLAN action.
SOLE executes the defined steps using the PLAN output, ensuring only the changes logged in the PLAN output are performed. This implies that you are aware of all the changes to be made.
AGGREGATE-DESTROY
Similar to AGGREGATE, the AGGREGATE-DESTROY action, executed by setting LIFECYCLE_ACTION to AGGREGATE-DESTROY, performs the PLAN-DESTROY and DESTROY actions in a single job.
PLAN-DESTROY
Similar to the PLAN action, when the LIFECYCLE_ACTION is set to PLAN-DESTROY, SOLE plans the execution steps to destroy one or more managed Snowflake objects. Using the existing state (if any), an execution plan to destroy these objects is generated and saved. The execution plan lists the changes it has detected to delete/drop the managed Snowflake objects.
The generated plan is provided to the DESTROY action.
DESTROY
When LIFECYCLE_ACTION is set to DESTROY, SOLE executes the actions specified by the PLAN-DESTROY action.
Using the output from PLAN-DESTROY, SOLE executes the defined deletion steps, ensuring only the changes logged by the PLAN-DESTROY action are performed. This also implies that the user is aware of all the changes to be made.
PLAN-ALL
As an alternative to setting up individual jobs for each object group and lifecycle action, you can configure a global plan action by setting the LIFECYCLE_ACTION to PLAN-ALL. Doing this performs the following actions for all object hierarchies sequentially and in a single job:
This reduces the number of pipeline jobs to one job that manages the Snowflake object lifecycle and generates a human-readable output.
Contrary to the AGGREGATE action, PLAN-ALL is coupled with Apply-ALL to separate the process allowing you to plan modifications before applying them. Check out Plan Snowflake setup job for more information.
As PLAN-ALL acts on all the object hierarchies, you don't have to specify the parameter LIFECYCLE_MANAGE_OBJECT. However, other parameters, such as CONFIGURATION_DIR, are still required.
APPLY-ALL
The APPLY-ALL action, executed by setting LIFECYCLE_ACTION to APPLY-ALL, performs the actions logged by PLAN-ALL in a single job.
Using the output from PLAN-ALL, SOLE executes the defined steps, ensuring only the changes logged by the PLAN-ALL action are performed. This also implies that you are aware of all the changes to be made. Check out Apply plan to Snowflake job for more information.