Skip to main content

Using Private Link with DataOps.live

This page will give you information and guidance on connecting to services external to DataOps using AWS PrivateLink, the steps necessary will depend on what you are trying to connect to.

AWS Private link with DataOps.live Develop and Runner !!shadow!!

To enable your users to connect to Snowflake through AWS PrivateLink, contact our Support team. After getting in touch, we can start creating resources in AWS and ask you to configure the Snowflake side.

First, you will use the AWS account ID and federated token we provided to perform actions in Snowflake. These actions will authorize the private link connection to your Snowflake account from the VPC we have created.

The details for enabling Private Link in your Snowflake account are in the AWS PrivateLink for Snowflake documentation. We have included some of the essential steps for this process below.

Using the AWS account ID and federated token, please run the following queries on your Snowflake account:

  1. Authorize the AWS account for your Snowflake Private Link endpoint
select SYSTEM$AUTHORIZE_PRIVATELINK ( '<account_id>' , '<federated_token>' );
  1. Run the following query and share part of the result with us. Doing so will confirm if the VPC has been authorized for Private Link endpoints.
select SYSTEM$GET_PRIVATELINK( '<account_id>' , '<federated_token>' );

We expect the returned result to contain Private link access authorized, meaning the operation has succeeded, and we can enable the rest of the setup from our side.

  1. Run the following query and share the result with us:
select SYSTEM$GET_PRIVATELINK_CONFIG();

You should get a result looking similar to:

output of GET_PRIVATELINK_CONFIG()
{
  "regionless-snowsight-privatelink-url": "<privatelink_org_snowsight_url>",
  "privatelink-account-name": "<account_identifier>",
  "privatelink-connection-ocsp-urls": "<client_redirect_ocsp_url_list>",
  "snowsight-privatelink-url": "<privatelink_region_snowsight_url>",
  "privatelink-internal-stage": "<privatelink_stage_endpoint>",
  "privatelink-account-url": "<privatelink_account_url>",
  "privatelink-connection-urls": "<privatelink_connection_url_list>",
  "regionless-privatelink-account-url": "<privatelink_org_account_url>",
  "privatelink-ocsp-url": "<privatelink_ocsp_url>",
  "privatelink-vpce-id": "<aws_vpce_id>",
  "privatelink-account-principal": "<aws_principal_arn>",
  "regionless-privatelink-ocsp-url": "<privatelink_org_ocsp_url>",
  "app-service-privatelink-url": "<privatelink_streamlit_url>"
}

We will need the results se we can appropriately configure DNS resolution from the platform to your Snowflake account with the given URLs.

Once you provided the configuration with the URLs, we will add the necessary DNS configuration to ensure your users get access to the Private Link connection. If you decide to include all members of your organization, no more steps are needed. Alternatively, let us know which members need access if you want to restrict access.

If at any point you want to turn this connection off, run:

select SYSTEM$REVOKE_PRIVATELINK( '<account_id>' , '<federated_token>' );

The DataOps Runner supports the same private networking solutions as Snowflake because the runner is deployed to your network, either on-premises or in your cloud.

To enable DataOps.live to access Snowflake using private networking, follow these steps:

  1. Go to the Snowflake documentation and follow the instructions on private networking.
  2. Make sure that the account identifier includes .privatelink as per the Snowflake account identifier documentation.
  3. Ensure that your runner is in the same private network that Snowflake authorized.
Last updated on