Before we look at how SOLE manages a Snowflake object's state, it is crucial to understand what state is in the context of DataOps and SOLE.
State (or local state) is the "condition" of all the Snowflake objects at a given point in time.
In other words, SOLE asks the following questions when inspecting the organization's Snowflake infrastructure:
- What objects exist?
- What do they look like, or how are they structured?
In practice, SOLE maps its configuration to the real world or the physical Snowflake ecosystem, allowing SOLE to use the existing state of the objects during subsequent DataOps pipeline runs.
This information is saved to a local state file and used during SOLE's
PLAN-DESTROY lifecycle actions and is saved in the persistent cache of the DataOps Runner's host system.
SOLE state management
The concept of state management builds on this definition of local state within the DataOps ecosystem. In summary, SOLE requires the local state information to function. And based on this requirement, state management is an integral part of the Snowflake Object Lifecycle Engine's processes.
Each environment and branch maintains its own set of states. Therefore, SOLE writes a local state file for each object group. Consequently, it is reasonable to conclude that, depending on the size of the organization and the number of Snowflake objects SOLE manages, there will be a significant number of state files to maintain and manage.
These state files are stored on the host system at the following path:
The values in angle brackets (
<>) translate into the following variables:
cache_directory: The host system cache directory
- The default is
- To retrieve the value for your DataOps Runner, see the volume mounts in the
- Refer to the DataOps Runner initial setup info
- The default is
dataops_project_name: The project name in lower-case
branch_name: The branch name in lower-case
resource_group: The object group name
State is maintained for all object groups, except for grants.
If the state file is corrupted or does not match the actual Snowflake infrastructure, pipeline failures can occur.
In such scenarios, you can reset the local state file to perform a fresh inspection of Snowflake. The reset can be triggered by the variable
LIFECYCLE_STATE_RESET set to
Resetting the local state file deletes the existing local state for the specified resource group and re-imports all managed Snowflake Objects.
The state file reset can be configured at the project level in
pipelines/includes/config/variables.yml to reset the state of all object groups. Or it can be configured at an individual object group level as a parameter to the
The individual object group state reset function is not supported in