Skip to main content

Key Pair Authentication

DataOps orchestrators provide for the use of key pair authentication as an enhanced authentication security protocol that is an alternative to the standard or basic authentication such as username and password.

In practice, key pair authentication support is enabled for the following orchestrators:

Before we look at how to configure key pair authentication, it is essential to note that the following key pairs are supported:

  • RSA private key in PEM format
  • Encrypted RSA private key in PKCS#8 format

This authentication method requires, as a minimum, a 2048-bit RSA key pair.

Configure Key Pair Authentication

While the detailed steps to configure key pair authentication are not part of the scope of this topic, it is worth summarizing the key pair authentication configuration workflow (as specified in the Snowflake documentation):

  1. Generate the private key as per Snowflake documentation
  2. Generate the public key as per Snowflake documentation
  3. Store the private and public keys securely using the vault configuration
  4. Assign the public key to a Snowflake user: see the Snowflake User SOLE Reference for rsa_public_key and rsa_public_key_2
  5. Verify the user's public key fingerprint
  6. Configure the Snowflake client to use key pair authentication

Supported Parameters

Before we look at an example key-pair vault configuration, it is vital to be aware of the following supported parameters; two required and two optional:

VariableRequiredDescription
DATAOPS_SNOWFLAKE_AUTHYesSet it to the KEY_PAIR value to enable key pair authentication support
DATAOPS_SNOWFLAKE_KEY_PAIRYesSpecify the key pair to be used
DATAOPS_SNOWFLAKE_PASSPHRASEOptionalThe passphrase when using an encrypted private key
PREVENT_KEY_PAIR_CLEANUPOptionalSkip key pair file removal on teardown process for MATE orchestrator
Configured key pairs are temporarily stored in a file used by

orchestrators and are removed before the job ends. As a consequence, by default, the key pair will not be available after the call to the /dataops script. To prevent deletion of the key pair files for MATE orchestrator one can set PREVENT_KEY_PAIR_CLEANUP. :::

Key Pair Vault Configuration

The following example describes what a key-pair vault configuration looks like if configured in the local /secrects/vault.yml:

/secrets/vault.yml
SNOWFLAKE:
SOLE:
ACCOUNT: <account> # DATAOPS_SOLE_ACCOUNT will be set from this local vault key
USERNAME: <username> # DATAOPS_SOLE_USERNAME will be set from this local vault key
ROLE: <role> # DATAOPS_SOLE_ROLE will be set from this local vault key

# DATAOPS_SNOWFLAKE_KEY_PAIR will be set from this local vault key
KEY_PAIR: "-----BEGIN RSA PRIVATE KEY-----
rmtbsfvsdcdtdxcvmyutr
athyjuhyktvdxdvcaerst
-----END RSA PRIVATE KEY-----"
# Optional passphrase if the key pair is encrypted
KEY_PASSPHRASE: XXXXXXX # DATAOPS_SNOWFLAKE_PASSPHRASE will be set from this local vault key

We recommend to use the secrects manager of your choice to store the SNOWFLAKE.SOLE.KEY_PAIR and SNOWFLAKE.SOLE.KEY_PASSPHRASE secret for higher security requirements.

Example Job Configuration

The variables section of the job or the config.yml file must be similar to the following code snippet:

My Job:
variables:
DATAOPS_SNOWFLAKE_AUTH: KEY_PAIR
DATAOPS_SNOWFLAKE_KEY_PAIR: DATAOPS_VAULT(SNOWFLAKE.SOLE.KEY_PAIR)
DATAOPS_SNOWFLAKE_PASSPHRASE: DATAOPS_VAULT(SNOWFLAKE.SOLE.KEY_PASSPHRASE)