Skip to main content

SOLE Lifecycle Actions

SOLE lifecycle actions are based on the requirement to manage the lifecycle (from create to destroy) of all managed Snowflake objects. In other words, these actions inform the SOLE Orchestrator what behaviors to execute. This content describes the following eight possible options for the LIFECYCLE_ACTION parameter:

Once we have completed discussing these eight lifecycle actions, the last part of this topic is to look at how to prevent deleting the production database and the Snowflake user utilized by SOLE.

AGGREGATE

As an alternative to setting up individual jobs for each object group and lifecycle action, an aggregate action, implemented by setting the LIFECYCLE_ACTION to AGGREGATE, can be configured to perform the following actions for all object hierarchies sequentially and in a single job:

This reduces the number of pipeline jobs to a single job instead of multiple jobs to manage the Snowflake object lifecycle.

tip

As AGGREGATE acts on all the object hierarchies, the parameter LIFECYCLE_MANAGE_OBJECT does not have to be specified. However, other parameters such as CONFIGURATION_DIR are still required.

COMPILE

When the LIFECYCLE_ACTION is set to COMPILE, SOLE generates the configuration required to initialize and manage Snowflake objects from the user-defined configurations. In other words, the COMPILE action tells SOLE to read all the YAML configuration files containing the user-defined configurations and generate a machine-readable configuration that SOLE can execute to enact the user-defined Snowflake configuration.

SOLE uses these configurations to perform validation and dependency resolution, generate resource configurations and syntax in the relevant stages, and resolve the grants and revokes on Snowflake Objects. See Access Management for more information on grants and revokes.

The following parameters are used to control the execution of the COMPILE action:

ParameterRequired/DefaultSet inDescription
CONFIGURATION_DIRREQUIREDSOLE OrchestratorThe directory path where the Snowflake configuration files are located
DATAOPS_PREFIXREQUIREDproject settingsThe prefix to be added to the account and database-level object hierarchies (except the default database)
DATAOPS_ENV_NAMEREQUIREDproject settingsThe suffix to be added to the account and database-level object hierarchies (except the default database)
DATAOPS_SOLE_DEBUGOptionalpipeline parametersThis parameter enables the debugging of SOLE without writing sensitive information to the logs.
DATAOPS_DEBUGOptionalpipeline parametersIf set, the DataOps debug logs are enabled

VALIDATE

When the LIFECYCLE_ACTION is set to VALIDATE, SOLE validates the resource configurations generated during the COMPILE stage.

PLAN

SOLE plans the execution steps to create or update the managed Snowflake objects when the LIFECYCLE_ACTION is set to PLAN.

Before generating a plan, SOLE tries to import all the specified Snowflake objects if they exist. This import helps initialize or update the local state for existing Snowflake objects used to generate the plan. An import summary is displayed during and after the completion of the import.

Due to aliases and multiple format support, a plan might include data-type alias changes (in-place changes) for Snowflake Objects.

Also, suppose the state file is corrupted or it contains a discrepancy, such as an object being managed in the state file but deleted from Snowflake. In this case, the state file can be removed and re-initialized from the current Snowflake object configuration. To do so, set the LIFECYCLE_STATE_RESET parameter to any value for the pipeline to re-initialize the local state for all object hierarchy groups. This parameter can also be initialized for any individual PLAN job to reset the state for a particular object or object group.

The generated plan is provided to the APPLY action.

APPLY

When the LIFECYCLE_ACTION is set to APPLY, SOLE executes the steps specified by the PLAN action.

Using the output from the PLAN, SOLE executes the defined steps, ensuring only the changes logged in the PLAN output are performed. This implies that the user is aware of all the changes to be made.

AGGREGATE-DESTROY

Similar to AGGREGATE, the AGGREGATE-DESTROY action, executed by setting LIFECYCLE_ACTION to AGGREGATE-DESTROY, performs the PLAN-DESTROY and DESTROY actions in a single job.

PLAN-DESTROY

Similar to the PLAN action, when the LIFECYCLE_ACTION is set to PLAN-DESTROY, SOLE plans the execution steps to destroy one or more managed Snowflake objects. Using the existing state (if any), an execution plan to destroy these objects is generated and saved. The execution plan lists the changes it has detected to delete/drop the managed Snowflake objects.

The generated plan is provided to the DESTROY action.

DESTROY

When LIFECYCLE_ACTION is set to DESTROY, SOLE executes the actions specified by the PLAN-DESTROY action.

Using the output from PLAN-DESTROY, SOLE executes the defined deletion steps, ensuring only the changes logged by the PLAN-DESTROY action are performed. This also implies that the user is aware of all the changes to be made.