Connection Parameters
To connect to the Snowflake account, SOLE requires the following details:
Account
This is the name of the Snowflake account where SOLE operates. Its value is set in the variable DATAOPS_SOLE_ACCOUNT. However, if there is no value in this variable, SOLE tries to obtain the value from the DataOps Vault at the path SNOWFLAKE.SOLE.ACCOUNT. As the Using the DataOps Vault section in the Core Concepts doc describes, SOLE can fetch the Snowflake account value from the DataOps Vault to prevent exposing credentials in the SOLE configuration.
The full account name and region, such as eu-west-2 must be defined in DATAOPS_SOLE_ACCOUNT. You can also use organizational short names and region-less account names in DATAOPS_SOLE_ACCOUNT.
You can execute the following query to get the regionless account name for your Snowflake account:
SELECT system$whitelist()
This query returns a JSON result with a key SNOWFLAKE_DEPLOYMENT_REGIONLESS with its value formatted as abcdef-qwerty.snowflakecomputing.com.
The complete abcdef-qwerty part can also be specified as the DATAOPS_SOLE_ACCOUNT.
Role
The role which runs the Snowflake queries is the role associated with the Snowflake user, defined in DATAOPS_SOLE_USERNAME. This value is stored in the variable DATAOPS_SOLE_ROLE and must be specified even if the role is the default role for the Snowflake user.
If this value is not located in this variable, SOLE tries to retrieve the value from the DataOps vault path SNOWFLAKE.SOLE.ROLE.
As described in the account section, SOLE can fetch the Snowflake role from the DataOps Vault to prevent exposing credentials in the SOLE configuration.
Warehouse
You can define the Snowflake warehouse used to run queries for SOLE grant management in the variable DATAOPS_SOLE_WAREHOUSE. This is an optional configuration. If it is not defined, SOLE grant management assumes that the user specified in DATAOPS_SOLE_USERNAME has a default warehouse set.
If no value is set in the variable DATAOPS_SOLE_WAREHOUSE, SOLE tries to retrieve it from the DataOps vault path SNOWFLAKE.SOLE.WAREHOUSE. If this vault path does not exist, SOLE grant management runs queries assuming the default warehouse is set.
It is possible to overwrite the DATAOPS_SOLE_WAREHOUSE variable for each -ci.yml file. The new value can point to a custom vault location and ensure that different warehouses will be used depending on the SOLE pipeline load. This affects only SOLE's usage of warehouses. Here is an example configuration:
---
variables:
DATAOPS_SOLE_WAREHOUSE: DATAOPS_VAULT(SNOWFLAKE.SOLE.WAREHOUSE_2)
Also, you can specify the name of the Snowflake warehouse SOLE uses to run hooks by defining the variable SNOWFLAKE_WAREHOUSE in your environment configuration. If it is not defined, an internal default warehouse is used. To use the same warehouse for hooks and grant management, assign SNOWFLAKE_WAREHOUSE the same value as DATAOPS_SOLE_WAREHOUSE used by grant management.
To apply a similar principle to other orchestrators, remember that the variable name may vary depending on the orchestrator in question. We recommend referring to the documentation specific to the relevant orchestrator for detailed information.
As described in the account section, SOLE can fetch the Snowflake warehouse from the DataOps vault to prevent exposing the credentials in the SOLE configuration.
Authentication
Password-based authentication
Username
This variable is the username that SOLE uses to connect to the account specified in DATAOPS_SOLE_ACCOUNT.
Its value is stored in the variable DATAOPS_SOLE_USERNAME. And if the value is not present in this variable, SOLE tries to retrieve the value from the DataOps vault path SNOWFLAKE.SOLE.USERNAME.
As described in the account section, SOLE can fetch the Snowflake username from the DataOps Vault to prevent exposing credentials in the SOLE configuration.
Password
The password belonging to the specified username is stored in DATAOPS_SOLE_PASSWORD.
If this value is not located in this variable, SOLE tries to retrieve the value from the DataOps vault path SNOWFLAKE.SOLE.PASSWORD.
As described in the account section, SOLE can fetch the Snowflake password from the DataOps Vault to prevent exposing credentials in the SOLE configuration.
Key pair-based authentication
SOLE implements Snowflake's key pair authentication rules and processes with the following as a foundation for SOLE and Snowflake key pair authentication:
In practice, key pair authentication support is enabled for SOLE and SOLE's grant management function.
Below is a quick summary of the supported parameters,
| Variable | Required | Description |
|---|---|---|
| DATAOPS_SNOWFLAKE_AUTH | Yes | Set it to the KEY_PAIR value to enable key pair authentication support |
| DATAOPS_SNOWFLAKE_KEY_PAIR | Yes | Specify the key pair to be used |
| DATAOPS_SNOWFLAKE_PASSPHRASE | Optional | The passphrase for encrypted key-pair |
To know more and how to configure it, see Key Pair Authentication