Warehouse

You can provide configuration to Snowflake Object Lifecycle Engine for the following operation with the warehouse:

Manage the lifecycle of new and existing warehouses
Manage the grants of a warehouse

Usage

note

We have introduced SOLE for Data Products as a new framework for SOLE to help you easily build an ecosystem of data products. The major difference is in how you define Snowflake objects in the configuration file. Rather than having a grouped collection of objects, SOLE for Data Products goes for modular, self-describing, and explicit object definition.
Learn more about SOLE for Data Products, currently available as a public preview.

We have also introduced Data products as an extra layer on top of the data product platform capabilities making managing data products easier than ever. Learn more about Data Products, currently available as a private preview.

Default Configuration
Data Products Configuration

warehouses:
  <warehouse-name>:
    <configuration-key>: <value>
    grants:
      <privilege>:
        - <role-name>
        - <role-name>

- warehouse:
    name: <warehouse-name>
    <configuration-key>: <value>
    grants:
      <privilege>:
        - rel(role.<role-name>)
        - rel(role.<role-name>)

Supported parameters

The engine supports the parameters listed below.

Configuration Key	Required/Optional	Data Types and Values	Description
`auto_resume`	Optional	Boolean	Specifies whether to automatically resume a warehouse when a SQL statement (e.g. query) is submitted to it
`auto_suspend`	Optional	Integer: Minimum allowed value is 40	Specifies the number of seconds of inactivity after which a warehouse is automatically suspended
`comment`	Optional	String	Specifies a comment for the warehouse
`deleted`	Optional	Boolean: `True` enables deletion prevention, `False` does nothing	Specifies what objects are allowed to be deleted
`environment`	Optional	String	Specifies the environment in which the warehouse is managed. Regex can be provided as well.
`grants`	Optional	Map: See Supported Warehouse Grants to Roles	List of Privileges and Roles to which privileges are granted on the current warehouse
`initially_suspended`	Optional	Boolean	Specifies whether the warehouse is created initially in the 'suspended' state. Note: Setting this value will cause the warehouse to be recreated on each SOLE run.
`manage_mode`	Optional	String: `all` (default), `none`, `grants`	Configures what properties to manage for the warehouse. See Changing Manage Mode before changing the value.
`max_concurrency_level`	Optional	Number	Specifies the concurrency level for SQL statements (i.e. queries and DML) executed by a warehouse
`max_cluster_count`	Optional	Integer	Specifies the maximum number of server clusters for the warehouse
`max_cluster_count`	Optional	Integer	Specifies the maximum number of server clusters for the warehouse
`namespacing`	Optional	String: `both` (default), `none`, `prefix`, `suffix`	Specifies whether prefix or suffix or both are to be added to warehouse name - doesn't apply to the default database.
`resource_monitor`	Optional	String	Specifies the name of a resource monitor that is explicitly assigned to the warehouse. Caution: The resource monitors set for the account cannot have any warehouses referencing them.
`scaling_policy`	Optional	String: `STANDARD`, `ECONOMY`	Specifies the policy for automatically starting and shutting down clusters in a multi-cluster warehouse running in auto-scale mode
`statement_timeout_in_seconds`	Optional	Number	Specifies the time, in seconds, after which a running SQL statement (query, DDL, DML, etc.) is canceled by the system
`statement_queued_timeout_in_seconds`	Optional	Number	Specifies the time, in seconds, a SQL statement (query, DDL, DML, etc.) can be queued on a warehouse before it is canceled by the system
`wait_for_provisioning`	Optional	Boolean	Specifies whether the warehouse, after being resized, waits for all the servers to provision before executing any queued or new queries
`warehouse_size`, `size`	Optional	String: `XSMALL` (default), `X-SMALL`, `SMALL`, `MEDIUM`, `LARGE`, `XLARGE`, `XLARGE`, `X2LARGE`, `2X-LARGE`, `XXXLARGE`, `X3LARGE`, `X-LARGE`, `X4LARGE`, `4X-LARGE`, `X5LARGE`, `5X-LARGE`, `X6LARGE`, `6X-LARGE`	Specifies the size of the virtual warehouse
`warehouse_type`	Optional	String: `STANDARD` (default), `SNOWPARK-OPTIMIZED`	Specifies the type of the virtual warehouse
`query_acceleration_max_scale_factor`	Optional	Number: `8` (default)	Specifies the maximum scale factor to free compute resources and accelerate the query workload in a warehouse. The scale factor is used as a multiplier based on warehouse size. The larger the warehouse, the larger the scale factor should be. Valid values are `0` to `100`.
`enable_query_acceleration`	Optional	Boolean	Specifies whether to enable the query acceleration service that speeds up the processing of queries while reducing their demand on the warehouse compute resources. See Using the Query Acceleration Service for more information,

Warehouse types

SOLE supports creating two Snowflake virtual warehouse types:

STANDARD (default)
SNOWPARK-OPTIMIZED

Snowflake virtual warehouses provide the resources to execute SQL queries and perform different operations. A warehouse is also defined by its size and the other properties you can set in SOLE to help control and automate the warehouse activities.

`SNOWPARK-OPTIMIZED` warehouse

SNOWPARK-OPTIMIZED warehouses are recommended for workloads requiring large memory. These warehouses provide 16x memory per node compared to a STANDARD Snowflake virtual warehouse.

Read more about Snowpark-optimized Warehouses in the Snowflake documentation.

The following warehouse_sizes are supported when creating a SNOWPARK-OPTIMIZED warehouse in SOLE:

MEDIUM
LARGE
XLARGE, X-LARGE
XXLARGE, X2LARGE, 2X-LARGE
XXXLARGE, X3LARGE, 3X-LARGE
X4LARGE, 4X-LARGE

Altering warehouse type

While the warehouse is in running status, you cannot alter the type. To alter the warehouse type:

Suspend the warehouse using the following command:
```
ALTER WAREHOUSE <warehouse-name> SUSPEND
```
In SOLE, modify the warehouse_type property to use the new value and run a pipeline.

note

You cannot alter the warehouse type to SNOWPARK-OPTIMIZED if its size is X-SMALL as SNOWPARK-OPTIMIZED warehouses don't support this size. Check the following topic for more detail.

Converting `STANDARD` to `SNOWPARK-OPTIMIZED`

The default warehouse size is set to X-SMALL for a STANDARD warehouse. Snowflake would use this default when creating the warehouse if not specified explicitly.

As X-SMALL is not supported with the SNOWPARK-OPTIMIZED warehouses, you must explicitly set the warehouse_size property to MEDIUM or something larger.

If you explicitly set the warehouse_size to X-SMALL or SMALL, you must update the value to MEDIUM or something larger to alter the warehouse type successfully.

In-Place changes

Using non-uppercase values for warehouse_size leads to in-place changes when running a PLAN or APPLY for a warehouse.

The value of warehouse_size is converted to the uppercase equivalent when the object is created in Snowflake and the value is saved in the local state as well. Due to this, when subsequent pipelines are run, the engine assumes that a change has to be made to convert the uppercase value to the specified size.

To reduce the number of in-place changes, we suggest using uppercase values for the warehouse Size, such as XSMALL or 2X-LARGE, instead of xsmall or 2x-large.

Supported warehouse grants to roles

Following are the privileges you can grant to roles in the warehouse definition:

ALL PRIVILEGES
MODIFY
MONITOR
USAGE
OPERATE
OWNERSHIP

ALL PRIVILEGES handling

When you define ALL PRIVILEGES in the SOLE configuration file, you grant all the privileges listed above to roles on this object except OWNERSHIP. However, the management of ALL PRIVILEGES in SOLE differs from its handling in Snowflake. See Handling ALL PRIVILEGES in SOLE for more information.

Examples

Warehouses with configured parameters

Default Configuration
Data Products Configuration

warehouses:
  DEMO_WAREHOUSE_2:
    comment: Test Comment
    warehouse_size: X-SMALL
    ## Please comment out max_cluster_count, min_cluster_count and scaling_policy if you are using a standard snowflake account.
    ## Read more on snowflake account types here: https://docs.snowflake.com/en/user-guide/intro-editions.html
    max_cluster_count: 2
    min_cluster_count: 1
    auto_suspend: 100
    auto_resume: true
    initially_suspended: true
    scaling_policy: standard
    statement_timeout_in_seconds: 23

  DEMO_WAREHOUSE_3:
    comment: Warehouse with Auto-resumed disabled
    warehouse_size: X3LARGE
    max_cluster_count: 5
    auto_suspend: 300
    auto_resume: false
    ## Please comment out max_cluster_count, min_cluster_count and scaling_policy if you are using a standard snowflake account.
    ## Read more on snowflake account types here: https://docs.snowflake.com/en/user-guide/intro-editions.html
    scaling_policy: ECONOMY

- warehouse:
    name: DEMO_WAREHOUSE_2
    comment: Test Comment
    warehouse_size: X-SMALL
    ## Please comment out max_cluster_count, min_cluster_count and scaling_policy if you are using a standard snowflake account.
    ## Read more on snowflake account types here: https://docs.snowflake.com/en/user-guide/intro-editions.html
    max_cluster_count: 2
    min_cluster_count: 1
    auto_suspend: 100
    auto_resume: true
    initially_suspended: true
    scaling_policy: standard
    statement_timeout_in_seconds: 23

- warehouse:
    name: DEMO_WAREHOUSE_3
    comment: Warehouse with Auto-resumed disabled
    warehouse_size: X3LARGE
    max_cluster_count: 5
    auto_suspend: 300
    auto_resume: false
    ## Please comment out max_cluster_count, min_cluster_count and scaling_policy if you are using a standard snowflake account.
    ## Read more on snowflake account types here: https://docs.snowflake.com/en/user-guide/intro-editions.html
    scaling_policy: ECONOMY

Warehouses with MODIFY and MONITOR privileges granted only to the role `SYSADMIN`

This would implicitly revoke all grants on privileges USAGE and OPERATE.

Default Configuration
Data Products Configuration

warehouses:
  DEMO_WAREHOUSE_4:
    comment: Test Comment
    warehouse_size: X-LARGE
    auto_resume: true
    initially_suspended: true
    grants:
      MODIFY:
        - SYSADMIN
      monitor:
        - SYSADMIN

- warehouse:
    name: DEMO_WAREHOUSE_4
    comment: Test Comment
    warehouse_size: X-LARGE
    auto_resume: true
    initially_suspended: true
    grants:
      MODIFY:
        - SYSADMIN
      monitor:
        - SYSADMIN

Warehouses all privileges revoked

Default Configuration
Data Products Configuration

warehouses:
  DEMO_WAREHOUSE_5:
    comment: Test Comment
    warehouse_size: x-large
    auto_resume: true
    initially_suspended: true
    grants:

- warehouse:
    name: DEMO_WAREHOUSE_5
    comment: Test Comment
    warehouse_size: x-large
    auto_resume: true
    initially_suspended: true
    grants:

Usage​

Supported parameters​

Warehouse types​

SNOWPARK-OPTIMIZED warehouse​

Altering warehouse type​

Converting STANDARD to SNOWPARK-OPTIMIZED​

In-Place changes​

Supported warehouse grants to roles​

Examples​

Warehouses with configured parameters​

Warehouses with MODIFY and MONITOR privileges granted only to the role SYSADMIN​

Warehouses all privileges revoked​