Key Pair Authentication

DataOps orchestrators provide the use of key pair authentication as an enhanced authentication security protocol that is an alternative to the standard or basic authentication such as username and password.

In practice, key pair authentication support is enabled for the following orchestrators:

• SOLE orchestrator
• MATE orchestrator

Before we look at how to configure key pair authentication, it is essential to note that the following key pairs are supported:

• RSA private key in PEM format
• Encrypted RSA private key in PKCS#8 format

This authentication method requires, as a minimum, a 2048-bit RSA key pair.

Configuring key pair authentication

While the detailed steps to configure key pair authentication are not part of the scope of this topic, it is worth summarizing the key pair authentication configuration workflow (as specified in the Snowflake documentation):

1. Generate the private key as per Snowflake documentation
2. Generate the public key as per Snowflake documentation
3. Store the private and public keys securely using the vault configuration
4. Assign the public key to a Snowflake user: see the Snowflake User SOLE Reference for rsa_public_key and rsa_public_key_2
5. Verify the user's public key fingerprint
6. Configure the Snowflake client to use key pair authentication

Supported parameters

Before we look at an example key-pair vault configuration, it is vital to be aware of the following supported parameters; two required and two optional:

Variable	Required	Description
DATAOPS_SNOWFLAKE_AUTH	Yes	Set it to the KEY_PAIR value to enable key pair authentication support
DATAOPS_SNOWFLAKE_KEY_PAIR	Yes	Specify the key pair to be used
DATAOPS_SNOWFLAKE_PASSPHRASE	Optional	The passphrase when using an encrypted private key
PREVENT_KEY_PAIR_CLEANUP	Optional	Skip key pair file removal on teardown process for MATE orchestrator

Configured key pairs are temporarily stored in a file used by

orchestrators and are removed before the job ends. As a consequence, by default, the key pair will not be available after the call to the /dataops script. To prevent deletion of the key pair files for MATE orchestrator one can set PREVENT_KEY_PAIR_CLEANUP. :::

Key pair vault configuration

The following example describes what a key-pair vault configuration looks like if configured in the local /secrects/vault.yml:

/secrets/vault.yml

SNOWFLAKE:
  SOLE:
    ACCOUNT: <account>                  # DATAOPS_SOLE_ACCOUNT will be set from this local vault key
    USERNAME: <username>                # DATAOPS_SOLE_USERNAME will be set from this local vault key
    ROLE: <role>                        # DATAOPS_SOLE_ROLE will be set from this local vault key

    # DATAOPS_SNOWFLAKE_KEY_PAIR  will be set from this local vault key
    KEY_PAIR: "-----BEGIN RSA PRIVATE KEY-----
rmtbsfvsdcdtdxcvmyutr
athyjuhyktvdxdvcaerst
-----END RSA PRIVATE KEY-----"
    # Optional passphrase if the key pair is encrypted
    KEY_PASSPHRASE: XXXXXXX             # DATAOPS_SNOWFLAKE_PASSPHRASE will be set from this local vault key

We recommend to use the secrects manager of your choice to store the SNOWFLAKE.SOLE.KEY_PAIR and SNOWFLAKE.SOLE.KEY_PASSPHRASE secret for higher security requirements.

Example job configuration

The variables section of the job or the config.yml file must be similar to the following code snippet:

My Job:
  variables:
    DATAOPS_SNOWFLAKE_AUTH: KEY_PAIR
    DATAOPS_SNOWFLAKE_KEY_PAIR: DATAOPS_VAULT(SNOWFLAKE.SOLE.KEY_PAIR)
    DATAOPS_SNOWFLAKE_PASSPHRASE: DATAOPS_VAULT(SNOWFLAKE.SOLE.KEY_PASSPHRASE)