Key Pair Authentication
DataOps orchestrators provide the use of key pair authentication as an enhanced authentication security protocol that is an alternative to the standard or basic authentication such as username and password.
In practice, key pair authentication support is enabled for the following orchestrators:
Before we look at how to configure key pair authentication, it is essential to note that the following key pairs are supported:
- RSA private key in PEM format
- Encrypted RSA private key in PKCS#8 format
This authentication method requires, as a minimum, a 2048-bit RSA key pair.
Configuring key pair authentication
While the detailed steps to configure key pair authentication are not part of the scope of this topic, it is worth summarizing the key pair authentication configuration workflow (as specified in the Snowflake documentation):
- Generate the private key as per Snowflake documentation
- Generate the public key as per Snowflake documentation
- Store the private and public keys securely using the vault configuration
- Assign the public key to a Snowflake user: see the
Snowflake User SOLE Reference for
rsa_public_key
andrsa_public_key_2
- Verify the user's public key fingerprint
- Configure the Snowflake client to use key pair authentication
Supported parameters
Before we look at an example key-pair vault configuration, it is vital to be aware of the following supported parameters; two required and two optional:
Variable | Required | Description |
---|---|---|
DATAOPS_SNOWFLAKE_AUTH | Yes | Set it to the KEY_PAIR value to enable key pair authentication support |
DATAOPS_SNOWFLAKE_KEY_PAIR | Yes | Specify the key pair to be used |
DATAOPS_SNOWFLAKE_PASSPHRASE | Optional | The passphrase when using an encrypted private key |
PREVENT_KEY_PAIR_CLEANUP | Optional | Skip key pair file removal on teardown process for MATE orchestrator |
orchestrators and are removed before the job ends. As a consequence, by default,
the key pair will not be available after the call to the /dataops
script. To
prevent deletion of the key pair files for MATE orchestrator one can set
PREVENT_KEY_PAIR_CLEANUP
. :::
Key pair vault configuration
The following example describes what a key-pair vault configuration looks like
if configured in the local /secrects/vault.yml
:
SNOWFLAKE:
SOLE:
ACCOUNT: <account> # DATAOPS_SOLE_ACCOUNT will be set from this local vault key
USERNAME: <username> # DATAOPS_SOLE_USERNAME will be set from this local vault key
ROLE: <role> # DATAOPS_SOLE_ROLE will be set from this local vault key
# DATAOPS_SNOWFLAKE_KEY_PAIR will be set from this local vault key
KEY_PAIR: "-----BEGIN RSA PRIVATE KEY-----
rmtbsfvsdcdtdxcvmyutr
athyjuhyktvdxdvcaerst
-----END RSA PRIVATE KEY-----"
# Optional passphrase if the key pair is encrypted
KEY_PASSPHRASE: XXXXXXX # DATAOPS_SNOWFLAKE_PASSPHRASE will be set from this local vault key
We recommend to use the
secrects manager of your
choice to store the SNOWFLAKE.SOLE.KEY_PAIR
and
SNOWFLAKE.SOLE.KEY_PASSPHRASE
secret for higher security requirements.
Example job configuration
The variables section of the job or the config.yml file must be similar to the following code snippet:
My Job:
variables:
DATAOPS_SNOWFLAKE_AUTH: KEY_PAIR
DATAOPS_SNOWFLAKE_KEY_PAIR: DATAOPS_VAULT(SNOWFLAKE.SOLE.KEY_PAIR)
DATAOPS_SNOWFLAKE_PASSPHRASE: DATAOPS_VAULT(SNOWFLAKE.SOLE.KEY_PASSPHRASE)